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1  Summary  and  Introduction 


We  have  touehed  upon  in  this  researeh  many  aspeets  of  distributed  eonstraint  problems  in  different  dis¬ 
tributed  environments  and  settings.  Our  results  provided  insights  to  some  of  the  intrinsie  issues  of  problem 
solving  in  distributed  environments.  The  final  produets  of  this  researeh  inelude  new  understanding  of 
phase  transitions  and  baekbones  of  eombinatorial  optimization  problems,  sueh  as  the  Traveling  Salesman 
Problem,  new  algorithms  for  eomplex  and  distributed  eonstraint  satisfaetion  and  optimization  problems, 
and  eorresponding  software. 

Our  researeh  has  been  mainly  foeused  on  five  topies.  We  briefly  summarize  the  main  findings  and 
results  in  the  following  five  paragraphs.  Eaeh  of  these  topies  will  be  diseussed  in  great  detail  in  a  separate 
seetion. 

The  first  topie  is  distributed  eonstraint  solving  and  low  overhead  distributed  eonstraint  algorithms.  We 
speeifieally  eonsider  eonstraint  problem  solving,  e.g.,  distributed  seheduling,  in  distributed  sensor  net¬ 
works.  We  study  sueh  distributed  algorithms  that  have  low  overhead  on  eomputation  and  communication. 
We  first  cast  such  problems  as  distributed  constraint  satisfaction  problems  (DisCSP)  and  distributed  con¬ 
straint  optimization  problems  (DisCOP)  and  model  them  as  distributed  multiple  coloring.  To  cope  with 
limited  resources  and  restricted  real-time  requirement,  we  consider  distributed  algorithms  that  have  low 
overhead  on  resource  consumption  and  high  anytime  performance.  To  meet  these  requirements,  we  study 
two  existing  DisCSP  algorithms,  distributed  stochastic  search  algorithm  (DSA)  and  distributed  breakout 
algorithm  (DBA),  for  solving  DisCOPs  and  the  distributed  scheduling  problems.  We  experimentally  show 
that  DSA  has  a  phase-transition  or  threshold  behavior,  in  that  its  solution  quality  degenerates  abruptly  and 
dramatically  when  the  degree  of  parallel  executions  of  distributed  agents  increases  beyond  some  critical 
value.  We  also  consider  the  completeness  and  complexity  of  DBA  for  graph  coloring.  We  show  that  DBA 
is  complete  on  acyclic  graphs  and  is  able  to  color  an  acyclic  graph  of  n  nodes  in  O(n^)  steps.  However, 
on  a  cyclic  graph,  DBA  may  never  terminate.  To  improve  DBAs  performance  on  coloring  cyclic  graphs, 
we  propose  two  stochastic  variations.  Finally,  we  directly  compare  DSA  and  DBA  for  solving  distributed 
multiple  coloring  and  distributed  scheduling  problems  in  sensor  networks.  The  results  show  that  DSA 
is  superior  to  DBA  when  controlled  properly,  having  better  or  competitive  solution  quality  and  signifi¬ 
cantly  lower  communication  cost  than  DBA.  Therefore,  DSA  is  the  algorithm  of  choice  for  our  distributed 
scheduling  problems. 

The  second  topic  is  negotiation  as  distributed  search.  Negotiation  is  one  of  the  main  mechanisms 
for  coordination  and  cooperation  in  distributed  environments.  However,  most  negotiation  protocols  are 
complex  and  their  features  are  difficult  to  characterize.  We  propose  a  general  experimental  approach  to 
analyzing  negotiation  strategies  using  distributed  search.  In  this  approach  we  first  formulate  the  problems 
that  negotiation  protocols  intend  to  solve  as  distributed  constraint  satisfaction/optimization  problems,  and 
then  capture  the  negotiation  protocols  as  distributed  search  algorithms.  By  analyzing  the  derived  search 
algorithms,  we  can  characterize  many  important  properties  of  the  negotiation  protocols.  In  this  paper,  we 
are  particularly  interested  in  the  properties  of  a  newly  developed  negotiation  protocol,  which  is  motivated 
by  distributed  sensor  network  applications,  including  its  completeness,  complexity,  convergence  rate,  and 
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scalability.  Although  the  idea  of  viewing  negotiation  as  distributed  search  is  not  completely  new,  in  this 
research  we  not  only  view  negotiation  as  distributed  search,  but  directly  apply  a  search  algorithm  to  reveal 
the  essential  features  of  a  negotiation  protocol  and  analyze  its  performance. 

The  third  topic  is  phase  transitions  and  backbones  of  combinatorial  optimization  problems,  the  Trav¬ 
eling  Salesman  Problem  in  particular.  In  recent  years,  phase  transitions  have  been  successfully  used  to 
analyze  combinatorial  optimization  problems,  characterize  their  typical-case  features  and  locate  the  hard¬ 
est  problem  instances.  In  this  research,  we  empirically  study  phase  transitions  of  the  asymmetric  Traveling 
Salesman,  an  NP-hard  combinatorial  optimization  problem  that  has  many  real-world  applications.  Using 
random  instances  of  up  to  1,500  cities,  we  show  that  many  properties  of  the  problem,  including  the  optimal 
tour  cost  and  backbone,  experience  sharp  transitions  as  the  precision  of  intercity  distances  increases  across 
a  critical  value.  Our  experimental  results  on  the  costs  of  the  Asymmetric  Traveling  Salesman  (ATSP)  tours 
and  assignment  problem  support  the  theoretical  result  that  the  asymptotic  cost  of  assignment  problem  is  tt^/G. 
In  addition,  we  show  that  the  average  computational  cost  of  the  well-known  branch-and-bound  subtour 
elimination  algorithm  for  the  problem  also  exhibits  a  threshold  behavior,  transitioning  from  easy  to  difficult  as 
the  distance  precision  increases.  These  results  answer  positively  an  open  question  regarding  the  existence  of 
phase  transitions  in  the  Traveling  Salesman,  and  provide  guidance  on  how  difficult  ATSP  problem  instances 
should  be  generated. 

The  fourth  topic  is  a  novel  metaheuristic  for  local  search  in  solving  Boolean  satisfiability.  Many 
constraint  problem  can  be  captured  as  Boolean  satisfiability  (SAT)  and  maximum  satisfiability  (Max- 
SAT),  which  are  difficult  combinatorial  problems  that  have  many  important  real-world  applications.  In 
this  research,  we  investigate  the  configuration  landscapes  of  local  minima  reached  by  the  WalkSAT  local 
search  algorithm,  one  of  the  most  effective  algorithms  for  SAT.  A  configuration  landscape  of  a  set  of 
local  minima  is  their  distribution  in  terms  of  quality  and  structural  differences  relative  to  an  optimal  or  a 
reference  solution.  Our  experimental  results  show  that  local  minima  from  WalkSAT  form  large  clusters, 
and  their  configuration  landscapes  constitute  big  valleys,  in  that  high  quality  local  minima  typically  share 
large  partial  structures  with  optimal  solutions.  Inspired  by  this  insight  into  WalkSAT  and  the  previous 
research  on  phase  transitions  and  backbones  of  combinatorial  problems,  we  propose  and  develop  a  novel 
method  that  exploits  the  configuration  landscapes  of  such  local  minima.  The  new  method  can  be  embedded 
in  a  local  search  algorithm,  such  as  WalkSAT,  to  improve  its  performance.  On  large  problem  instances 
from  a  SAT  library  (SATLIB),  the  backbone  guided  WalkSAT  algorithm  finds  satisfiable  solutions  to  SAT 
problems  with  higher  probabilities  than  WalkSAT,  and  obtains  better  solutions  on  Max-SAT  problems, 
improving  solution  quality  by  20%  on  average. 

The  last  topic  is  a  set  of  effective  methods  for  complex  scheduling  problems.  We  consider  complex 
scheduling  problems  that  can  be  captured  as  optimization  under  hard  and  soft  constraints.  The  objective  of 
such  an  optimization  problem  is  to  satisfy  as  many  hard  constraints  as  possible  and  meanwhile  to  minimize 
a  penalty  function  of  unsatisfied  soft  constraints.  We  present  an  efficient  local  search  algorithm  for  these 
problems  which  improves  upon  WalkSAT(oip),  a  WalkSAT-based  local  search  algorithm  for  overconstrained 
problems  represented  in  integer  programs.  We  introduce  three  techniques  to  the  WSAT(oip)  algorithm  to 
extend  its  capability  and  improve  its  performance:  backbone  guided  biased  moves  to  drive  the  search  to 
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the  regions  in  seareh  spaee  where  high-quality  and  optimal  solutions  reside;  sampling-based  aspiration 
seareh  to  reduee  seareh  eost  and  make  anytime  solutions  available  over  the  eourse  of  the  seareh;  and 
dynamie  parameter  tuning  to  dynamieally  adjust  the  key  parameters  of  the  algorithm  to  make  it  robust 
and  flexible  for  various  applieations.  Our  experimental  results  on  large-seale  erew  seheduling,  basketball 
tournament  seheduling  and  progressive  party  seheduling  show  that  the  new  improved  algorithm  ean  find 
better  solutions  with  less  eomputation  than  WSAP(oip). 

Most  of  the  researeh  deseribed  in  this  report  has  been  published  in  many  eonferenee  and  journal  papers, 
which  include  those  of  [100,  110,  111,  115,  116,  117,  118,  119,  120]. 

The  rest  of  the  report  will  describe  these  topics  and  results  in  detail.  As  these  topics  are  relatively 
independent,  we  draw  conclusions  for  each  of  them,  at  the  end  of  their  corresponding  sections. 
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2  Low  Overhead  Distributed  Constraint  Algorithms 


In  recent  years,  various  micro-electro-mechanical  systems  (MEMS)  devices,  such  as  sensors  and  actuators 
with  some  information  processing  capabilities  embedded  within,  have  been  developed  and  deployed  in 
many  real-world  applications  [85,  93].  For  example,  one  application  that  we  have  been  involved  with  is 
detecting  and  tracking  mobile  objects  in  dynamic  and  real-time  environments,  where  distributed  sensors 
must  cooperatively  monitor  an  area  to  detect  new  objects  and  some  of  the  sensors  have  to  sense  and 
measure  at  the  same  time  in  order  to  triangulate  a  target.  Such  distributed  sensor  environments  were 
specific  research  platforms  in  the  Autonomous  Negotiating  Teams  (ANTs)  domains.  In  such  distributed 
applications,  there  are  logical  restrictions  among  the  actions  of  the  sensors  and  actuators  and  restrictions 
on  available  computational  and  communication  resources  as  well  as  restrictions  on  energy  consumption. 

Multi-agent  system  technology  can  play  a  critical  role  in  developing  large-scale  networked,  embedded 
systems  using  such  smart  devices,  by  providing  frameworks  for  building  and  analyzing  such  systems. 
Due  to  the  real-time  nature  of  many  applications  and  limited  computational  resources  on  the  devices,  e.g., 
slow  CPUs  and  small  memories,  the  key  to  large-scale,  real-time  MEMS  is  the  mechanism  that  the  smart 
devices  (or  agents)  use  to  make  viable  distributed  decisions  in  restricted  time  with  limited  computational 
resources.  Therefore,  the  methods  for  distributed  constraint  problem  solving  are  also  important  tools  for 
such  real-time  decision  making  in  distributed  environments. 

In  contrast  to  the  current  research  on  sensor  networks,  which  has  largely  focused  on  building  ad  hoc, 
mobile  communication  networks  (e.g.,  [40,  121]),  this  line  of  research  on  applying  multi-agent  system 
technology  and  constraint  problem-solving  techniques  has  a  paramount  importance  to  the  overall  success 
of  embedded  sensor  networks  by  addressing  the  key  issue  of  resource  allocation  at  the  application  level. 

We  have  been  developing  large-scale  sensor  networks  controlled  by  distributed  multi-agent  systems 
for  the  applications  of  mobile  object  detection  and  tracking  problem  and  the  vibration  dampening  in  the 
avionics  domain.  We  place  an  agent  on  top  of  a  sensor,  which  runs  on  a  battery  and  has,  among  other 
things,  a  slow  CPU,  a  small  memory  and  a  wireless  communication  unit.  The  sensors/agents  need  to  be 
programmed  in  such  a  way  that  they  can  collaboratively  carry  out  a  task,  which  no  single  sensor/agent  can 
do  on  its  own.  In  this  research,  we  are  particularly  interested  in  low-overhead  methods  for  solving  DisCSPs 
and  DisCOPs.  By  low-overhead  methods  we  mean  those  in  which  limited  communication  among  agents 
is  needed  and  agents  make  their  decisions  without  global  knowledge  of  the  system  and  without  knowing 
the  actions  of  the  other  agents. 

In  this  research,  we  study  distributed  stochastic  algorithm  (DSA)  [25,  27,  67]  and  distributed  breakout 
algorithm  (DBA)  [79,  103,  105],  two  existing  algorithms  for  DisCSP,  for  the  purpose  of  solving  our  dis¬ 
tributed  scheduling  problems  in  sensor  networks.  In  other  words,  we  apply  and  extend  these  algorithms  to 
solving  DisCOP,  which  is  much  more  difficult  than  DisCSP. 

Note  that  DSA  and  DBA  are  incomplete  in  that  they  do  not  guarantee  to  find  satisfying  solutions 
fo  DisCSPs  and  fhus  may  nol  find  opfimal  solutions  fo  DisCOPs  eifher.  Neverfheless,  we  choose  fhese 
algorifhms  for  fwo  reasons.  Firsf,  compared  fo  complefe  disfribufed  algorifhms,  such  as  fhose  of  [103, 104], 
DSA  and  DBA  have  low  compufafional  and  communicafion  cosf.  An  agenf  in  fhese  algorifhms  only 
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needs  information  loeal  to  itself  and  information  of  neighboring  agents  eonneeted  through  the  underlying 
eonstraint  graphs.  Therefore,  the  eomputational  eost  is  low.  These  algorithms  also  do  not  have  to  reeord 
many  previous  states  visited,  so  that  memory  requirement  is  low  as  well.  Seeond,  as  we  will  see  later  in 
this  seetion,  these  algorithms  have  good  anytime  performanee,  finding  good  approximate  solutions  quiekly. 
Even  if  they  are  not  guaranteed  the  optimality  of  solutions  that  they  provide,  sueh  heuristie  algorithms  are 
suffieient  for  finding  good  approximafe  solufions  in  real-fime  environmenfs  where  fhe  fargefing  objeefives 
may  also  ehange  over  time,  making  optimal  solufions  less  imporfanf. 

In  addition  fo  fhe  above  imporfanf  eharaeferisfies  fo  serve  fhe  needs  of  some  applieafions  in  sensor 
nefworks,  DSA  and  DBA  also  have  unique  feafures  in  resolving  eonfiiefs  among  disfribufed  agenfs.  DBA 
infroduees  priorifies  among  agenfs’  aefions  for  eonfiief  resolufion,  while  DSA  uses  randomness  fo  possi¬ 
bly  avoid  eonfliels.  Moreover,  as  shown  in  fhis  seefion,  DSA  has  a  fhreshold  behavior,  in  whieh  solufion 
qualify  degrades  abrupfly  and  dramafieally  as  fhe  randomness  for  eonfiief  resolutions  inereases  beyond  a 
erifieal  value,  and  DSA  beeomes  eomplefe  and  has  a  low  polynomial  time  eomplexify  for  disfribufed  mul- 
fiple  eoloring  problems  when  fhe  underlying  eonsfrainf  graphs  are  aeyelie.  However,  on  ofher  eonsfrainf 
graph  sfruefures,  ineluding  fhe  eonsfrainf  graphs  eonsfruefed  from  our  motivating  applieafions,  DSA  is 
superior  fo  DBA,  generally  finding  better  solufions  wifh  less  time.  Overall,  our  experimenfal  resulfs  show 
fhaf  DSA  is  a  good  ehoiee  of  algorifhm  for  our  sensor  nefwork  applieafions. 

This  seefion  is  organized  as  follows.  A  defailed  aeeounf  of  fhe  seheduling  problems  is  given  in  See¬ 
fion  2.1.  We  formulale  fhe  problems  as  weighfed  mulfi-eoloring  problems  in  Seefion  2.2.  We  fhen  deseribe 
DSA  and  analyze  ifs  fhreshold  behavior  in  Seefion  2.3.  We  eonsider  DBA  and  ifs  eomplefeness  and  eom¬ 
plexify  for  graph  eoloring  in  Seefion  2.4.  We  also  diseuss  infrodueing  randomness  fo  improve  DBA’s 
performanee.  We  eompare  fhe  performanee  of  DSA  and  DBA  on  disfribufed  mulfiple  graph  eoloring  and 
our  disfribufed  seheduling  problem  of  finding  fhe  shorfesf  sean  sehedule  for  mobile  objeef  defeefion  in 
Seefion  2.5.  We  diseuss  relafed  work  in  Seefion  2.6,  and  finally  eonelude  in  Seefion  2.7. 

Early  resulfs  of  fhis  seefion  have  appeared  in  [111,  116,  118,  120]. 

2.1  Distributed  Constraint  Satisfaction  and  Optimization  in  Sensor  Networks 

To  sef  fhe  sfage  for  our  invesfigafion  of  disfribufed  algorifhms  for  disfribufed  eonsfrainf  problems,  we  firsl 
deseribe  our  applieafion  problems  in  sensor  nefworks. 

Defeefing  and  fraeking  mobile  objeefs  in  large  open  environmenfs  is  an  imporfanf  fopie  fhaf  has  many 
real  applieafions  in  differenf  domains,  sueh  as  avionies,  surveillanee  and  robof  navigafion.  We  are  devel¬ 
oping  sueh  an  objeef  defeefing  and  fraeking  sysfem  using  MEMS  sensors,  eaeh  of  whieh  operafes  under 
resfriefed  energy  sourees,  i.e.,  batteries,  and  has  a  small  amounf  of  memory  and  resfriefed  eompufafional 
power.  In  a  fypieal  applieafion  of  our  sysfem,  a  eolleefion  of  small  Doppler  sensors  are  seaffered  in  an  open 
area  fo  defeef  possible  foreign  objeefs  moving  info  fhe  region.  Eaeh  sensor  ean  sean  and  defeef  an  objeef 
wifhin  a  fixed  radius.  However,  fhe  overall  defeefing  area  of  a  sensor  is  divided  info  fhree  equal  seefors, 
and  fhe  sensor  ean  only  operafe  in  one  seefor  af  any  given  time.  The  sensors  ean  eommunieafe  wifh  one 
anofher  fhrough  radio  eommunieafion.  The  radio  ehannel  is  nof  reliable,  however,  in  fhaf  a  signal  may 
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get  lost  due  to,  for  instanee,  a  eollision  of  signals  from  multiple  sensors,  or  distorted  due  to  environment 
noises.  Moreover,  switehing  from  one  seanning  seetor  to  another  seetor  and  sending  and  reeeiving  radio 
signals  take  time  and  energy.  To  save  energy,  a  sensor  may  turn  itself  off  oeeasionally  or  periodieally  if 
doing  so  does  not  reduee  system  performanee. 

The  overall  system  operates  to  aehieve  two  eonflieting  goals,  to  deteet  foreign  objeets  as  many  and  as 
quiekly  as  possible  and  meanwhile  to  preserve  energy  as  mueh  as  possible  so  as  to  prolong  the  system’s 
lifetime.  The  overall  problem  is  thus  to  find  a  distributed  sean  sehedule  to  optimize  an  objeetive  funetion 
that  balanees  the  above  two  eonflieting  goals. 

One  of  our  goals  here  is  to  develop  a  sealable  sensor  network  for  objeet  deteetion  in  that  the  system 
response  time  for  de  tee  ting  a  new  objeet  does  not  degenerate  when  the  system  size  inereases.  To  this 
end,  we  plaee  an  agent  on  top  of  eaeh  sensor  and  make  the  agent  as  autonomous  as  possible.  This  means 
that  an  agent  will  not  rely  on  information  of  the  whole  system,  rather  the  loeal  information  ineluding 
its  neighboring  sensors  or  agents.  These  agents  also  try  to  avoid  unneeessary  eommunieation  so  as  to 
minimize  system  response  time.  Therefore,  eomplex  eoordination  and  reasoning  methods  are  not  suitable. 

We  set  forth  to  apply  simple,  low-overhead  methods  for  solving  this  distributed  seheduling  problem. 
We  assume  that  the  sensors  are  stationary,  in  that  they  will  not  move  after  being  plaeed  in  an  opera¬ 
tion  environment.  We  eonsider  two  distributed  algorithms,  distributed  breakout  and  distributed  stoehastie 
seareh,  that  were  developed  earlier  to  solve  distributed  eonstraint  satisfaetion  problems.  Our  objeetives 
are  twofold.  First,  we  want  to  understand  how  these  two  algorithms  eompare  with  eaeh  other  on  a  real- 
world  applieation  sueh  as  our  sean  seheduling  problem.  Seeond,  we  want  to  identify  the  right  applieation 
eondition  and  parameters  of  the  algorithm  that  we  will  use  based  on  the  evaluation. 

2.2  Constraint  Models  in  Distributed  Multiple  Coloring 

Our  first  step  to  address  the  distributed  seheduling  problems  is  to  model  them  as  distributed  eonstraint 
satisfaetion  and  optimization  problems.  Indeed,  the  eore  problems  in  our  applieations  diseussed  in  the 
previous  seetion  ean  be  eaptured  as  distributed  multi-eoloring  problems. 

In  the  mobile  objeet  deteetion  problem,  a  sensor  needs  to  sean  its  three  seetors  as  often  as  possible. 
However,  to  save  energy,  two  neighboring  sensors  should  try  to  avoid  seanning  a  common  area  covered  by 
two  overlapping  sensor  sectors  at  the  same  time  since  one  sensor’s  sensing  of  an  area  is  sufficient  to  detect 
possible  objects  in  the  area.  The  objective  is  to  find  a  sequence  of  sensing  actions  so  that  each  sensor  sector 
can  be  scanned  at  least  once  within  a  minimal  period  of  time.  In  other  words,  the  objective  is  a  cyclic  scan 
schedule  in  which  each  sector  is  scanned  at  least  once  with  the  constraint  of  minimizing  the  energy  usage. 

As  all  agents  execute  the  same  procedure,  scanning  actions  on  different  sensors  take  approximately 
the  same  amount  of  time.  Assume  that  the  sensors  are  synchronized*,  an  assumption  that  can  be  lifted  by 
a  synchronization  method^  [94].  Therefore,  the  problem  can  be  viewed  as  coloring  a  weighted  graph  so 

*The  algorithms  we  will  consider  shortly,  DSA  and  DBA,  are  synchronous.  It  has  also  been  observed  that  under  certain 
conditions  asynchronized  actions  may  actually  improve  the  performance  of  a  distributed  algorithm  [26]. 

^Indeed,  DBA  and  its  synchronization  mechanism  have  been  implemented  and  tested  on  a  real-time  middleware  using  Object 
Request  Broker  framework 
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that  the  total  weight  of  violated  eonstraints  is  minimized.  Here,  a  node  eorresponds  to  a  sensor  or  agent, 
a  link  between  two  nodes  represents  the  eonstraint  of  a  shared  region  between  two  agents,  and  the  weight 
eaptures  the  area  of  a  common  region.  The  larger  a  shared  region  is,  the  more  energy  will  be  wasted  if 
two  sensors  scan  the  shared  region  at  the  same.  Moreover,  each  color  corresponds  to  a  time  slot  in  which 
a  sector  is  scanned  and  the  total  number  of  colors  represents  the  scanning  cycle  length  or  the  number  of 
time  slots  required  to  scan  each  of  the  sectors  of  all  sensors.  A  node  must  have  at  least  one  color  so  that 
the  corresponding  sector  is  scanned  at  least  once,  and  may  also  have  multiple  colors  so  that  the  sector  may 
be  sensed  multiple  times  to  increase  system’s  object  detection  rate. 

We  further  consider  a  sensor  as  an  hypernode  of  three  nodes,  each  of  which  corresponds  to  a  sector  of 
the  sensor.  Within  an  hypernode,  two  nodes  cannot  share  a  color  since  two  sectors  cannot  be  scanned  at  the 
same  time.  In  other  words,  the  colors  assigned  to  the  nodes  within  an  hypernode  are  mutually  exclusive. 
This  is  a  hard  constraint  that  cannot  be  violated.  Furthermore,  not  all  available  colors  must  be  used  as 
a  sensor  may  sometimes  turn  itself  off.  A  node  within  a  hypernode  may  also  be  constrained  by  a  node 
of  another  hypernode  if  the  two  corresponding  sectors  share  a  common  region.  This  constraint  may  be 
violated,  resulting  in  wasted  energy.  Overall,  we  are  looking  for  such  a  coloring  that  satisfies  all  hard 
constraints  within  hypernodes  and  minimizes  the  weights  of  violated  constraints  among  hypernodes  so  as 
to  reduce  the  overall  conflicts  of  scanning  common  regions  among  overlapping  sectors. 

We  can  model  the  distributed  scan  scheduling  problem  as  follows.  Let  G{V,  E)  be  an  undirected 
graph  with  a  set  of  hypernodes  V  and  a  set  of  edges  E.  Each  hypernode  v  ^V,  which  represents  a  sensor, 
consists  of  k  nodes,  vi,V2,-  "  >  which  model  k  sectors  of  the  sensor.  (In  our  application,  we  have 
k  =  3.)  An  edge  between  nodes  Ui  ^  u  and  Vj  G  f  is  denoted  as  (rt*,  Vj)  G  E,  and  Ui  and  Vj  are  called 
neighbors.  Two  hypernodes  are  neighbors  if  any  pair  of  their  nodes  are  neighbors.  The  weight  of  an  edge 
{ui,Vj),  denoted  as  w{ui,  vj),  is  the  area  of  a  common  region  covered  by  sectors  of  sensors  u  and  v.  Every 
hypernode  is  given  a  total  T  available  colors  that  can  be  used  and  a  sensor  activation  ratio  a  <  1.  |r| 
corresponds  to  a  schedule  length,  a  captures  the  frequency  of  how  often  a  sensor  should  be  active,  and 
1  —  a  is  the  frequency  determining  how  often  a  sensor  should  turn  itself  off  to  save  energy.  Given  the 
total  available  colors  T,  the  weighted  multi-coloring  problem  is  to  color  the  nodes  following  these  criteria. 
Eirst,  exactly  [aT]  colors  are  used  within  a  hypernode.  This  means  that  a  node  may  have  more  than  one 
color  if  the  available  colors  are  more  than  the  nodes  in  a  hypernode,  i.e.,  [aT]  >  k.  Second,  every  node 
must  have  at  least  one  color  and  no  two  nodes  within  a  hypernode  can  share  a  color.  Third,  minimizing 
conflicts  between  pairs  of  nodes  in  different  hypernodes,  so  the  total  weight  of  violations  going  across 
hypernodes  is  minimized.  When  no  conflict  is  allowed  or  a  minimal  level  of  allowed  conflicts  is  specified, 
the  overall  problem  is  to  find  the  minimal  number  of  allowed  colors  T  and  such  a  schedule  satisfying  the 
criteria. 

The  distortion  and  damage  detection  problem  can  also  be  captured  by  a  constraint  model.  Scheduling 
the  signaling  activities  of  the  ping  nodes  can  be  formulated  as  a  distributed  graph  coloring  problem.  A 
color  here  corresponds  to  a  time  slot  in  which  a  ping  node  sends  out  signals.  The  number  of  colors  is 
therefore  the  length  in  time  units  of  a  schedule.  The  problem  is  to  find  a  shortesf  schedule  such  that  the 
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pinging  signals  do  not  interfere  with  one  another  in  order  to  inerease  damage  deteetion  response  time  and 
reduee  the  amount  of  wasted  energy.  The  problem  is  equivalent  to  finding  the  ehromatie  number  of  a  given 
eonstraint  graph,  whieh  eorresponds  to  the  minimal  worst-ease  response  time  and  a  eoloring  of  the  graph 
within  the  overall  system  response  time. 

In  short,  this  damage  deteetion  problem  is  a  distributed  eonstraint  satisfaetion/optimization  problem 
with  variables  and  eonstraints  distributed  among  agents.  Colleetively  the  agents  find  a  solufion  fo  minimize 
an  objeefive  funefion,  whieh  is  fhe  number  of  violafed  eonsfrainfs. 

2.3  Distributed  Stochastic  Search 

In  Ibis  section,  we  sfudy  disfribufed  stochastic  search  algorithm.  In  fact,  this  is  not  a  single  algorithm,  but 
a  family  of  distributed  algorithms,  as  we  will  see  shortly. 

2.3.1  The  algorithm 

Distributed  stochastic  algorithm  (DS A)  is  uniform  [94] ,  in  that  all  processes  are  equal  and  have  no  iden¬ 
tities  to  distinguish  one  another.  It  is  also  synchronous  in  principle  [94],  in  that  all  processes  proceed  in 
synchronized  steps  and  in  each  step  it  sends  and  receives  (zero  or  more)  messages  and  then  performs  local 
computations,  i.e.,  changing  local  state.  Note  that  synchronization  in  DSA  is  not  crucial  since  it  can  be 
achieved  by  a  synchronization  mechanism  [94]. 

The  idea  of  DSA  and  its  variations  is  simple  [25,  27,  67,  83].  After  an  initial  step  in  which  the  agents 
pick  random  values  for  their  variables,  they  go  through  a  sequence  of  steps  until  a  termination  condition 
is  met.  In  each  step,  an  agent  sends  its  current  state  information,  i.e.,  its  variable  value  in  our  case,  to 
its  neighboring  agents  if  it  changed  its  value  in  the  previous  step,  and  receives  the  state  information  from 
the  neighbors.  It  then  decides,  often  stochastically,  to  keep  its  current  value  or  change  to  a  new  one.  The 
objective  for  value  changing  is  to  possibly  reduce  violated  constraints.  A  sketch  of  DSA  is  in  Algorithm  1. 

The  most  critical  step  of  DSA  is  for  an  agent  to  decide  the  next  value,  based  on  its  current  state  and  its 
believed  states  of  the  neighboring  agents.  If  the  agent  cannot  find  a  new  value  to  improve  its  current  state, 
it  will  not  change  its  current  value.  If  there  exists  such  a  value  that  improves  or  maintains  state  quality,  the 
agent  may  or  may  not  change  to  the  new  value  based  on  a  stochastic  scheme. 

Table  1  lists  five  possible  strategies  for  value  change,  leading  to  five  variations  of  the  DSA  algorithm. 
In  DSA- A,  an  agent  will  change  its  value  only  when  the  state  quality  can  be  improved.  DSA-B  is  the  same 
as  DSA-A  except  that  an  agent  may  also  change  its  value  if  there  is  a  violated  constraint  and  changing  its 
value  will  not  degrade  state  quality.  DSA-B  is  expected  to  have  a  better  performance  than  DSA-A  since 
by  reacting  stochastically  when  the  current  state  cannot  be  improved  directly  (A  =  0  and  there  exists  a 
conflict),  the  violated  constraint  may  be  satisfied  in  the  next  step  by  the  value  change  at  one  of  the  agents 
involved  in  the  constraint.  Thus,  DSA-B  will  change  value  more  often  and  has  a  higher  degree  of  parallel 
actions  than  DSA-A. 

Furthermore,  DSA-C  is  more  aggressive  than  DSA-B,  changing  value  even  if  the  state  is  at  a  local 
minima  where  there  exist  no  conflict  but  another  value  leading  to  a  state  of  the  same  quality  as  the  current 


Algorithm  1  Sketch  of  DSA,  executed  by  all  agents. 
Randomly  choose  a  value 
while  (no  termination  condition  is  met)  do 
if  (a  new  value  is  assigned)  then 
send  the  new  value  to  neighbors 
end  if 

collect  neighbors’  new  values,  if  any 
select  and  assign  the  next  value  (See  Table  1) 

end  while 


Algo. 

A  >  0 

C,  A  =  0 

no  C,  A  =  0 

DSA-A 

V  with  p 

- 

- 

DSA-B 

V  with  p 

V  with  p 

- 

DSA-C 

V  with  p 

V  with  p 

V  with  p 

DSA-D 

V 

V  with  p 

- 

DSA-E 

V 

V  with  p 

V  with  p 

Table  1:  Next  value  selection  in  DSAs.  Here  C  stands  for  conflict,  A  is  the  best  possible  conflict  reduction 
between  two  steps,  v  the  value  giving  A,  and  p  a  probability  to  change  the  current  value,  which  represents 
the  degree  of  parallel  executions,  and  means  no  value  change.  Notice  that  when  A  >  0  there  must  be 
a  conflict. 

one.  An  agent  in  DSA-C  may  move  to  such  an  equal-quality  value  in  the  next  step.  It  is  hoped  that  by 
moving  to  another  value,  an  agent  gives  up  its  current  value  that  may  block  any  of  its  neighbors  to  move 
to  a  better  state.  Therefore,  the  overall  quality  of  the  algorithm  may  improve  by  introducing  this  equal- 
quality  action  at  a  single  node.  The  actual  effects  of  this  move  remain  to  be  examined,  which  is  one  of  the 
objectives  of  this  research. 

Parallel  to  DSA-B  and  DSA-C,  we  have  two  more  aggressive  variations.  DSA-D  (DSA-E)  extends 
DSA-B  (DSA-C)  by  allowing  an  agent  to  move,  deterministically,  to  a  new  value  as  long  as  it  can  improve 
the  current  state  (A  >  0).  These  variations  make  an  agent  more  greedily  self  centered  in  that  whenever 
there  is  a  good  move,  it  will  take  it. 

Notice  that  the  level  of  activities  at  an  agent  increase  from  DSA-A,  to  DSA-B  and  to  DSA-C,  and  from 
DSA-D  to  DSA-E.  The  level  of  activities  also  reflects  the  degree  of  parallel  executions  among  neighboring 
processes.  When  the  level  of  local  activities  is  high,  so  is  the  degree  of  parallel  executions. 

To  change  the  degree  of  parallel  executions,  an  agent  may  switch  to  a  different  DSA  algorithm,  or 
change  the  probability  p  that  controls  the  likelihood  of  updating  its  value  if  the  agent  attempts  to  do  so. 
This  probability  controls  the  level  of  activities  at  individual  agents  and  the  degree  of  parallel  executions 
among  neighboring  processes.  One  major  objective  of  this  research  is  to  investigate  the  effects  of  this 
control  parameter  on  the  performance  of  DSA  algorithms. 

The  termination  conditions  and  methods  to  detect  them  are  complex  issues  of  their  own.  We  will  adopt 
a  termination  detection  algorithm  [94]  in  a  later  stage.  In  our  current  implementation,  we  terminate  DSAs 
after  a  fixed  number  of  steps.  This  simple  determination  method  serves  the  basic  needs  of  the  current 
research,  i.e.,  experimentally  investigating  the  behavior  and  performance  of  these  algorithms,  one  of  the 
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main  focuses  of  this  section. 


2.3.2  Phase  Transitions 

DSAs  are  stochastic,  in  that  they  may  behave  differently  even  if  all  conditions  are  equal.  We  are  interested 
in  their  typical  or  statistical  behavior  at  an  equilibrium  state  when  the  behavior  of  the  algorithms  does  not 
seem  to  change  dramatically  from  one  step  to  the  next.  We  are  specifically  interested  in  the  relationship 
between  the  degree  of  parallel  executions,  controlled  by  the  probability  p  (cf.  Table  1),  and  the  performance 
of  the  algorithms,  including  their  solution  quality  and  communication  costs. 

It  turns  out  that  the  performance  of  DSAs  may  experience  phase  transitions  on  some  constraint  struc¬ 
tures  when  the  degree  of  parallelism  increases.  Phase  transitions  refer  to  a  phenomenon  of  a  system  in 
which  some  global  properties  change  rapidly  and  dramatically  when  a  control  or  order  parameter  goes 
across  a  critical  value  [14,  44].  A  simple  example  of  a  phase  transition  is  water  changing  from  liquid  to  ice 
when  the  temperature  drops  below  the  freezing  point.  For  the  problem  of  interest  here,  the  system  property 
is  DSAs  performance  (solution  quality  and  communication  cost)  and  the  order  parameter  is  the  probability 
p  that  controls  the  degree  of  parallel  executions  of  the  agents. 

Solution  quality:  We  experimentally  investigate  DSAs’  phase-transition  behavior  on  grids,  random 
graphs  and  trees.  In  our  experiments,  we  used  different  networks,  including  grids,  which  appear  in  our 
motivating  application,  and  graphs  and  trees.  We  considered  graph  coloring  problems,  by  varying  the 
connectivity  of  the  structures  and  the  number  of  colors  used,  we  are  able  to  generate  underconstrained, 
critically  constrained  and  overconstrained  problem  instances.  In  the  following  discussions,  we  will  focus 
on  grid  and  graph  structures. 

Starting  from  random  initial  colorings,  we  let  the  algorithms  run  for  a  large  number  of  steps,  to  the 
point  where  they  seem  to  reach  an  equilibrium,  i.e.,  the  overall  coloring  quality  does  not  change  signifi¬ 
cantly  from  one  step  to  the  next.  We  then  measure  the  solution  quality,  in  terms  of  the  number  of  constraints 
violated.  In  our  experiments,  we  measure  the  performance  at  1,000  steps;  longer  executions,  such  as  5,000 
and  10,000  steps,  exhibit  almost  the  same  results. 

The  distributed  algorithms  were  simulated  on  one  machine  using  a  discrete  event  simulation  method  [103]. 
In  this  method,  an  agent  maintains  a  step  counter,  equivalent  to  a  simulated  clock.  The  counter  is  increased 
by  one  after  the  agent  has  executed  one  step  of  computation,  in  which  it  sends  its  state  information,  if  nec¬ 
essary,  receives  neighbors’  messages,  and  carries  out  local  computation.  The  overall  solution  quality  is 
measured,  at  a  particular  time  point,  by  the  total  number  of  constraints  violated,  and  the  communication 
cost  is  measured  by  the  total  number  of  messages  sent. 

We  varied  the  degree  of  parallel  executions,  the  probability  p  in  Table  1,  and  examined  the  quality  of 
the  colorings  that  DSAs  can  provide.  The  solution  quality  indeed  exhibits  phase-transition  behavior  on 
grid  and  graph  structures  as  the  degree  of  parallelism  increases. 

Grids  We  generate  grids  of  various  sizes,  including  20  x  20,  40  x  40  and  60  x  60  grids,  and  use  different 
number  of  colors,  ranging  from  two  to  eight.  In  order  to  study  how  DSAs  will  scale  up  to  large  problems, 
we  simulate  infinitely  large  grids.  We  remove  the  grid  boundaries  by  connecting  the  nodes  on  the  top  to 
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Figure  1:  Solution  quality  phase  transitions  on  2-eoloring  grids;  fe  =  4  (left)  and  k  =  8  (right). 


Figure  2:  Solution  quality  phase  transitions  on  grids;  k  =  8  using  4  eolors  (left)  and  5  eolors  (right). 


those  on  the  bottom  as  well  as  the  nodes  on  the  left  to  those  on  the  right  of  the  grids.  We  also  ehange  the 
degree  of  eonstrainedness  by  ehanging  the  number  of  neighbors  that  a  node  may  have.  For  example,  on  a 
degree  k  =  4  grid,  each  node  has  four  neighbors,  one  each  to  the  top,  the  bottom,  the  left  and  the  right. 
Similarly,  on  a  degree  k  =  8  grid,  each  node  has  eight  neighbors,  one  each  to  the  top  left,  top  right,  bottom 
left  and  bottom  right  in  addition  to  the  four  neighbors  in  a  A:  =  4  grid. 

Figure  1  shows  the  total  numbers  of  constraint  violations  after  1 ,00  steps  of  the  algorithms  using  two 
colors  on  20  x  20  grids  with  fc  =  4  (Figure  l(left))  and  k  =  8  (Figure  l(right)).  Each  data  point  of  the 
figure  is  averaged  over  1,000  random  initial  colorings.  Note  that  the  results  from  larger  grids,  such  as 
40  X  40  grids,  follow  almost  identical  patterns  as  in  Figure  1 . 

The  figures  show  fhaf  DSAs’  phase-fransifion  behavior  is  confrolled  by  fhe  degree  of  parallelism,  ex- 
cepf  DSA-A  on  grids  wifh  fc  =  4.  The  fransifions  are  typically  very  sharp.  For  example,  as  Figure  l(lefl) 
shows,  fhe  solufion  qualify  of  DSA-B  and  DSA-C  decreases  abrupfly  and  dramafically  when  fhe  proba- 
bilify  p  increases  above  0.8.  More  imporfanfly  and  surprisingly,  afler  fhe  fransifion,  fhe  solufion  qualify  is 
even  worse  fhan  a  random  coloring.  The  average  solution  quality  of  random  colorings  corresponds  fo  fhe 
poinf  p  =  0  on  fhe  DSA-B  and  DSA-C  curves  in  fhe  figure.  This  indicafes  fhaf  fhe  degree  of  parallel  exe- 
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cutions  should  be  controlled  under  a  certain  level  in  order  for  the  algorithms  to  have  a  good  performance. 
Furthermore,  the  transitions  start  earlier  for  DSA-D  and  DSA-E.  Although  DSA-A,  the  most  conservative 
algorithm,  does  not  show  phase  transitions  on  grids  of  /c  =  4,  its  average  solution  quality  is  much  worse 
than  that  of  DSA-B,  because  it  may  be  easily  trapped  in  local  minima. 

The  degree  of  parallelism  and  the  constrainedness  of  the  underlying  network  structures  also  interplay. 
Grids  with  /c  =  4  are  2-colorable  while  grids  with  /c  =  8  are  not,  and  are  thus  overconstrained.  The  results 
shown  in  Figure  1  indicate  that  the  phase  transitions  appear  sooner  on  overconstrained  problems  than  on 
underconstrained  problems.  Even  the  most  conservative  DSA-A  also  experiences  a  phase  transition  when 
k  =  8.  The  most  aggressive  ones,  DSA-D  and  DSA-E,  always  performs  worse  than  a  random  coloring  on 
this  overconstrained  grid. 

A  coloring  problem  becomes  easier  if  more  colors  are  used,  since  it  is  less  constrained  to  find  a  satisfy¬ 
ing  color  in  the  next  step.  However,  the  phase-transition  behavior  persists  even  when  the  number  of  colors 
increases.  Eigure  2  shows  the  results  on  grids  with  k  =  8  using  4  and  5  colors.  Notice  that  the  curves  in 
the  4-color  figure  and  the  curves  for  3  colors  (not  shown  here)  follow  similar  patterns  as  in  the  case  for  2 
colors  in  Eigure  1  (right). 

Graphs  The  phase  transitions  of  DSAs  persist  on  graphs  as  well,  and  follow  similar  patterns  as  in  the 
grid  cases.  We  conducted  experiments  on  random  graphs.  We  generate  graphs  with  400  and  800  nodes 
and  average  node  connectivity  equal  to  /c  =  4  and  k  =  8.  A  graph  is  generated  by  adding  edges  to 
randomly  selected  pairs  of  nodes.  These  two  types  of  graphs  are  used  to  make  a  correspondence  to  the 
grid  structures  of  A:  =  4  and  k  =  8  mentioned  before.  We  also  generated  random  trees  with  depth  four  and 
average  branching  factors  k  =  A  and  k  =  8. 

Eigure  3  shows  the  results  on  graphs  with  k  =  A  and  k  =  8  using  2  colors,  and  Eigure  4  the  results 
on  graphs  with  k  =  8  using  4  and  5  colors.  Each  data  point  is  an  average  of  1,000  random  instances. 
The  solution  quality  is  also  measured  after  1,000  steps  of  executions.  As  all  the  figures  show,  fhe  phase 
fransifions  on  random  graphs  have  similar  pafferns  as  fhose  on  grids.  Therefore,  fhe  discussions  on  fhe  grids 
apply  in  principle  fo  random  graphs.  We  need  fo  mention  fhaf  on  graphs,  fhe  mosf  aggressive  algorifhms, 
DSA-D  and  DSA-E,  do  nol  perform  very  well  under  all  degrees  of  parallel  executions.  This,  combined 
wifh  fhe  resulfs  on  grids,  leads  fo  fhe  conclusion  fhaf  DSA-D  and  DSA-E  should  nol  be  used. 

Trees  There  is  no  phase  Iransilion  observed  on  random  frees  in  our  lesls.  All  DSAs  perform  poorly  on 
2-coloring,  in  comparison  wifh  Iheir  performance  on  grids  and  graphs.  This  seems  fo  be  counlerinluilive 
since  frees  have  fhe  simplesl  slruclures  among  all  fhese  nelwork  slruclures.  One  explanalion  is  fhaf  DSAs 
may  be  easily  Irapped  info  local  minima.  Since  frees  are  always  2-colorable,  we  are  able  fo  easily  creale 
local  minima  in  which  none  of  DSAs  can  escape. 

Communication  cost:  We  have  so  far  focused  on  DSAs’  solution  quality  without  paying  any  attention 
to  their  communication  costs.  Communication  in  a  sensor  network  has  an  inherited  delay  and  could  be 
unreliable  in  many  situations.  Therefore,  communication  cost  of  a  distributed  algorithm  is  an  integral  part 
of  its  overall  performance.  It  is  desirable  to  keep  communication  cost  as  low  as  possible. 

In  fact,  the  communication  cost  of  a  DS  A  algorithm  goes  hand-in-hand  with  its  solution  quality.  Recall 
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Probability  of  Parallelisni 


Figure  3:  Solution  quality  phase  transitions  on  2-coloring  graphs;  k  =  A  (left)  and  k  =  8  (right). 


Figure  4:  Solution  quality  phase  transitions  on  graphs;  k  =  8  using  4  colors  (left)  and  5  colors  (right). 


that  an  agent  will  send  a  message  to  its  neighbors  after  it  changed  its  value  (cf.  Algorithm  1  and  Table  1). 
In  DS  A-A,  DSA-B  and  DSA-D,  an  agent  may  change  its  value  if  there  is  a  conflict,  and  will  not  do  so  if  it  is 
currently  at  a  state  of  a  local  minimum,  while  in  DSA-C  and  DSA-E,  an  agent  may  probabilistically  change 
its  value  at  a  local  minimum  state.  Therefore,  in  general  the  communication  cost  at  a  node  will  go  down  if 
the  agent  moves  to  a  better  state,  and  go  up  otherwise.  As  a  result,  the  overall  communication  cost  will  also 
follow  similar  trends.  If  the  solution  quality  of  DSA  improves  over  time,  so  does  its  communication  cost. 
Therefore,  the  communication  cost  is  also  controlled  by  the  degree  of  parallel  executions  of  the  agents. 
The  higher  the  parallel  probability  p  is,  the  higher  the  communication  cost  will  be. 

We  verified  this  prediction  by  experiments  on  grids  and  graphs,  using  the  same  problem  instances  as 
used  for  analyzing  solution  quality.  Figure  5  shows  the  communication  cost  on  grids  with  k  =  8  using  4 
colors  (left)  and  5  colors  (right)  after  1,000  steps.  Comparing  these  these  figures  with  those  in  Figure  2, 
it  is  obvious  that  solution  quality  and  communication  cost  follow  identical  patterns.  Furthermore,  the 
communication  cost  on  graphs  (not  shown  here)  follows  similar  patterns  as  those  on  grids. 
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Figure  5:  Communication  phase  transitions  on  grids  with  k  =  8  using  4  colors  (left)  and  5  colors  (right). 


2.4  Breakout  and  Distributed  Breakout 

Distributed  Breakout  algorithm  was  based  on  a  centralized  predecessor.  To  better  understand  the  dis¬ 
tributed  algorithm,  we  start  with  the  centralized  version.  We  then  study  the  strength  and  weakness  of  DBA 
for  graph  coloring. 

2.4.1  The  algorithm 


Algorithm  2  Sketch  of  DBA 

set  the  local  weights  of  constraints  to  one 
value  <—  a  random  value  from  domain 
while  (no  termination  condition  met)  do 
exchange  value  with  neighbors 
WR  <—  BestPossibleWeightReductionO 
send  WR  to  neighbors  and  collect  their  WRs 
if  (WR  >  0)  then 

if  (it  has  the  biggest  improvement  among  neighbors)  then 
value  <—  the  value  that  gives  WR 

end  if 
else 

if  (no  neighbor  can  improve)  then 

increase  violated  constraints’  weights  by  one 

end  if 
end  if 
end  while 


The  breakout  algorithm  [79]  is  a  local  search  method  equipped  with  an  innovative  scheme  of  escaping 
local  minima  for  CSR  Given  a  CSP,  the  algorithm  first  assigns  a  weight  of  one  to  all  constraints.  It  then 
picks  a  value  for  every  variable.  If  no  constraint  is  violated,  the  algorithm  terminates.  Otherwise,  it  chooses 
a  variable  that  can  reduce  the  total  weight  of  the  unsatisfied  constraints  if  its  value  is  changed.  If  such  a 
weight-reducing  variable-value  pair  exists,  the  algorithm  changes  the  value  of  a  chosen  variable.  The 
algorithm  continues  the  process  of  variable  selection  and  value  change  until  no  weight-reducing  variable 
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can  be  found.  At  that  point,  it  reaches  a  local  minimum  if  a  constraint  violation  still  exists.  Instead  of 
restarting  from  another  random  initial  assignment,  the  algorithm  tries  to  escape  from  the  local  minimum 
by  increasing  the  weights  of  all  violated  constraints  by  one  and  proceeds  as  before.  This  weight  change 
will  force  the  algorithm  to  alter  the  values  of  some  variables  to  satisfy  the  violated  constraints. 

Centralized  breakout  can  be  extended  to  distributed  breakout  algorithm  (DBA)  [103,  105].  Without 
loss  of  generality,  we  assign  an  agent  to  a  variable,  and  assume  that  all  agents  have  unique  identifiers.  Two 
agents  are  neighbors  if  they  share  a  common  constraint.  An  agent  communicates  only  with  its  neighbors. 
At  each  step  of  DBA,  an  agent  exchanges  its  current  variable  value  with  its  neighbors,  computes  the  pos¬ 
sible  weight  reduction  if  it  changes  its  current  value,  and  decides  if  it  should  do  so.  To  avoid  simultaneous 
variable  changes  at  neighboring  agents,  only  the  agent  having  the  maximal  weight  reduction  has  the  right 
to  alter  its  current  value.  If  ties  occur,  the  agents  break  the  ties  based  on  their  identifiers.  The  above  pro¬ 
cess  of  DBA  is  skefched  in  Algorifhm  2.  For  simplicify,  we  assume  each  sfep  is  synchronized  among  fhe 
agenfs.  This  assumption  can  be  tiffed  by  a  synchronizafion  mechanism  [94]. 

In  fhe  description  of  [103,  105],  each  agenf  also  mainfains  a  variable,  called  my-termination-counter 
(MTC),  fo  help  deled  a  possible  termination  condition.  At  each  step,  an  agent’s  MTC  records  the  diam¬ 
eter  of  a  subgraph  centered  around  the  agent  within  which  all  the  agents’  constraints  are  satisfied.  For 
inslances,  an  agenl’s  MTC  is  zero  if  one  of  ils  neighbors  has  a  violated  consfrainf;  if  is  equal  fo  one  when 
ils  immediafe  neighbors  have  no  violafion.  Therefore,  if  fhe  diameter  of  fhe  consfrainf  graph  is  known  fo 
each  agenf,  when  an  agenl’s  MTC  is  equal  fo  fhe  known  diameter,  DBA  can  lerminale  wilh  fhe  currenl 
agenf  values  as  a  satisfying  solulion.  However,  MTCs  may  never  become  equal  fo  fhe  diameter  even  if  a 
solulion  exisls.  There  are  cases  in  which  fhe  algorifhm  is  nol  complele  in  fhal  if  cannol  guaranlee  fo  find 
a  solution  even  if  one  exisls.  Such  a  worsl  case  depends  on  fhe  slruclure  of  a  problem,  a  topic  of  fhe  nexl 
section.  We  do  nol  include  fhe  MTC  here  to  keep  our  descriplion  simple. 

If  is  worlh  poinling  oul  fhal  fhe  node,  or  agenf,  idenlifiers  are  nol  essential  fo  fhe  algorifhm.  They  are 
only  used  fo  sel  up  a  priority  belween  Iwo  compeling  agenfs  for  lie  breaking.  As  long  as  such  priorilies 
exisls,  node  idenlifiers  are  nol  needed. 

2.4.2  Completeness  and  Complexity 

Distributed  algorithms  are  generally  incomplete  in  that  they  cannot  guarantee  to  find  a  solution  even  if  one 
exists.  It  is  also  generally  difficult  to  show  the  completeness  of  a  distributed  algorithm  and  to  analyze  its 
complexity.  In  this  section,  we  consider  the  completeness  and  complexity  of  DBA  on  a  special  class  of 
problems,  i.e.,  distributed  graph  coloring.  Here,  the  complexity  is  defined  as  DBA’s  number  of  synchro¬ 
nized  distributed  steps.  In  one  step,  value  changes  at  different  nodes  are  allowed  while  one  variable  can 
change  its  value  at  most  once.  We  also  use  variables,  nodes  and  agents  interchangeably  in  our  discussion. 

Acyclic  graphs:  First  notice  that  acyclic  graphs  are  2-colorable.  Thus,  coloring  acyclic  graphs  is  less 
constrained  with  more  colors.  Therefore,  it  is  sufficient  to  consider  acyclic  graphs  with  two  colors.  To 
simplify  our  discussion,  we  first  consider  chains,  which  are  special  acyclic  graphs.  The  results  on  chains 
will  also  serve  as  a  basis  for  trees. 
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Chains  We  will  refer  to  the  eombination  of  variable  values  and  eonstraint  weights  as  a  problem  state,  or 
state  for  short.  A  solution  of  a  eonstraint  problem  is  a  state  with  no  violated  eonstraint.  We  say  two  states 
are  adjacent  if  DBA  ean  move  from  one  state  to  the  other  within  one  step. 

Lemma  2,1  DBA  will  not  visit  the  same  problem  state  more  than  once  when  2-coloring  a  chain. 

Proof:  Assume  the  opposite,  i.e.,  DBA  ean  visit  a  state  twiee  in  a  proeess  as  follows,  Sx  ^  Sy  ^  ^ 

Sz  ^  Sx-  Obviously  no  eonstraint  weight  is  allowed  to  inerease  at  any  state  on  this  eyele.  Suppose  that 
node  X  ehanges  its  value  at  state  Sx  to  resolve  a  eonfliet  C  involving  x.  In  the  worst  ease  a  new  eonfliet  at 
the  other  side  of  the  node  will  be  ereated.  C  is  thus  “pushed”  to  the  neighbor  of  x,  say  y.  Two  possibilities 
exist.  First,  C  is  resolved  at  y  or  another  node  along  the  ehain,  so  that  no  state  eyele  will  form.  Seeond, 
C  returns  to  x,  eausing  x  to  ehange  its  value  baek  to  its  previous  value.  Sinee  nodes  are  ordered,  i.e.,  they 
have  prioritized  identifiers,  violations  may  only  move  in  one  direetion  and  C  eannot  return  to  x  from  y 
without  ehanging  a  eonstraint  weight.  This  means  that  C  must  move  baek  to  x  from  another  path,  whieh 
eontradiets  the  faet  that  the  strueture  is  a  ehain.  □ 

Lemma  2.2  DBA  can  increase  a  constraint  weight  to  at  most  [n/2j  when  coloring  a  chain  ofn  variables 
using  at  least  two  colors. 

Proof:  The  weight  of  the  first  eonstraint  on  the  left  of  the  ehain  will  never  ehange  and  thus  remain  at  one, 
sinee  the  left  end  node  ean  always  ehange  its  value  to  satisfy  its  only  eonstraint.  The  weight  of  the  seeond 
eonstraint  on  the  left  ean  inerease  to  two  at  the  most.  When  the  weight  of  the  seeond  eonstraint  is  two 
and  the  seeond  eonstraint  on  the  left  is  violated,  the  seeond  node  will  always  ehange  its  value  to  satisfy 
the  seeond  eonstraint  beeause  it  has  a  higher  weight  than  the  first  eonstraint.  This  will  push  the  violation 
to  the  left  end  node  and  foree  it  to  ehange  its  value  and  thus  resolve  the  eonfliet.  This  argument  ean  be 
induetively  applied  to  the  other  internal  nodes  and  eonstraints  along  the  ehain.  In  faet,  it  ean  be  applied  to 
both  ends  of  the  ehain.  So  the  maximal  eonstraint  weight  on  the  ehain  will  be  [n/2j .  □ 

Immediate  eorollaries  of  this  lemma  are  the  best  and  worst  arrangements  of  variable  identifiers.  In  the 
best  ease,  the  end  nodes  of  the  ehain  should  be  most  aetive,  always  trying  to  satisfy  the  only  eonstraint, 
and  resolving  any  eonfliet.  Therefore,  the  end  nodes  should  have  the  highest  priority,  followed  by  their 
neighbors,  and  so  on  to  the  middle  of  the  ehain.  The  worst  ease  is  simply  the  opposite  of  the  best  ease;  the 
end  nodes  are  most  inaetive  and  have  the  lowest  priority,  followed  by  their  neighbors,  and  so  on. 

Theorem  2.1  DBA  terminates  in  at  most  n?  steps  with  a  solution,  if  it  exists,  or  with  an  answer  of  no 
solution,  if  it  does  not  exist,  when  coloring  a  chain  ofn  variables  using  at  least  two  colors. 

Proof:  As  a  ehain  is  always  2-eolorable,  the  eombination  of  the  above  lemmas  gives  the  result  for  a  ehain 
with  nodes  of  domain  sizes  at  least  two.  It  is  possible,  however,  that  no  solution  exists  if  some  variables 
have  fixed  values  or  eolors.  In  this  ease,  it  is  easy  to  ereate  a  eonfliet  between  two  nodes  with  domain  size 
one,  whieh  will  never  be  resolved.  As  a  result,  the  weights  of  the  eonstraints  between  these  two  nodes 
will  be  raised  to  n.  If  eaeh  agent  knows  the  ehain  length  n,  DBA  ean  be  terminated  when  a  eonstraint 
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weight  is  more  than  n.  (In  faet,  the  ehain  length  ean  be  eomputed  in  0(n)  steps  as  follows.  An  end  node 
first  sends  number  1  to  its  only  neighbor.  The  neighboring  node  adds  one  to  the  number  reeeived  and  then 
passes  the  new  number  to  the  other  neighbor.  The  number  reaehed  at  the  other  end  of  the  ehain  is  the  ehain 
length,  whieh  ean  be  subsequently  disseminated  to  the  rest  of  the  ehain.  The  whole  proeess  takes  2n  steps.) 
Furthermore,  a  node  needs  at  most  n  —  1  steps  to  inerease  a  eonstraint  weight.  This  worst  ease  oeeurs  when 
a  ehain  eontains  two  variables  at  two  ends  of  the  ehain  whieh  have  the  lowest  priorities  and  unity  domain 
sizes  so  neither  of  them  ean  ehange  its  value.  On  sueh  a  ehain,  a  eonfliet  ean  be  pushed  around  between 
the  two  end  nodes  many  time.  Every  time  a  eonfliet  reaehes  an  end  node,  the  node  inereases  the  eonstraint 
weight  to  push  the  eonfliet  baek.  Sinee  a  eonstraint  weight  will  be  no  more  than  n,  the  result  follows.  □ 

A  signifieant  implieation  of  these  results  is  a  termination  eondition  for  DBA  for  eoloring  a  ehain.  If 
DBA  does  not  find  a  solution  in  steps,  it  ean  terminate  with  the  eonelusion  that  no  solution  exists.  This 
new  termination  eondition  and  DBAs  original  termination  eondition  of  my-termination-eounter  guarantee 
DBA  to  terminate  on  a  ehain. 

Trees  The  key  to  the  proof  for  eoloring  ehain  and  tree  struetures  is  that  no  eyele  exists  in  an  aeyelie  graph, 
so  that  the  same  eonfliet  eannot  return  to  a  node  without  inereasing  a  eonstraint  weight. 

The  arguments  on  the  maximal  eonstraint  weight  for  eoloring  ehains  hold  for  general  aeyelie  graphs 
or  trees.  First  eonsider  the  ease  that  eaeh  variable  has  a  domain  size  at  least  two.  In  an  aeyelie  graph,  an 
arbitrary  eonstraint  (link)  C  eonneets  two  disjoint  aeyelie  graphs,  Gi  and  G2.  Assume  Gi  and  G2  have  m 
and  n2  nodes,  respeetively,  and  rii  <  n2-  Then  the  maximal  possible  weight  W  on  G  eannot  be  more  than 
ni,  whieh  is  proven  induetively  as  follows.  If  the  node  v  assoeiated  with  C  is  the  only  node  of  Gi,  then 
the  elaim  is  true  sinee  v  ean  always  aeeommodate  G.  If  Gi  is  a  ehain,  then  the  arguments  for  Lemma  2.2 
apply  direetly  and  the  maximal  possible  weight  of  a  eonstraint  is  the  number  of  links  the  eonstraint  is 
away  from  the  end  variable  of  Gi.  If  n  is  the  only  node  in  Gi  that  eonneets  to  more  than  one  eonstraint 
in  Gi,  whieh  we  eall  a  branehing  node,  then  a  eonfliet  at  C  may  be  pushed  into  Gi  when  the  weight  of 
G  is  greater  than  the  sum  of  the  weights  of  all  eonstraints  in  Gi  linked  to  v,  whieh  is  at  most  equal  to  the 
number  of  nodes  of  Gi.  The  same  arguments  equally  apply  when  v  is  not  the  only  branehing  node  of  Gi. 
Therefore,  the  maximal  eonstraint  weight  is  bounded  by  n. 

The  worst-ease  eomplexity  ean  be  derived  similarly.  A  worst  ease  oeeurs  when  all  end  variables  of 
an  aeyelie  graph  have  fixed  values,  so  that  a  eonfliet  may  never  be  pushed  out  of  the  graph.  A  eonstraint 
weight  ean  be  bumped  up  by  one  after  a  eonfliet  has  traveled  from  an  end  node  to  other  end  nodes  and 
baek,  within  at  most  n  steps. 

Based  on  these  arguments,  we  have  the  following  result. 

Theorem  2.2  DBA  terminates  in  at  most  n?  steps  with  either  an  optimal  solution,  if  it  exists,  or  an  answer 
of  no  solution,  if  it  does  not  exist,  when  coloring  an  acyclic  graph  with  n  nodes  with  at  least  two  colors. 

The  above  eompleteness  result  ean  be  direetly  translated  to  eentralized  breakout  algorithm,  leading  to 
its  eompleteness  of  eoloring  aeyelie  graphs  as  well.  Moreover,  sinee  eaeh  step  in  DBA  is  equivalent  to 
n  steps  in  the  eentralized  algorithm,  eaeh  of  whieh  examines  a  distinet  variable,  the  eomplexity  result  on 
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chain  length  number  of  nodes 

Figure  6:  The  number  of  steps  taken  by  DBA  on  chains  with  the  best  and  worst  variable  identifier  arrange¬ 
ments  (left)  and  on  trees  with  worst  identifier  arrangements  (right). 


DBA  also  means  that  the  worst-case  complexity  of  the  centralized  algorithm  is  O(n^).  These  analytical 
results  reveal  the  superiority  of  centralized  breakout  algorithm  and  DBA  over  conventional  local  search 
methods  of  coloring  acyclic  graphs,  including  the  distributed  stochastic  algorithm  discussed  in  Section  2.3, 
which  are  not  complete  even  on  a  chain. 

Our  experimental  results  also  show  that  the  number  of  steps  taken  by  DBA  is  much  smaller  than  the 
upper  bound,  as  shown  in  Figure  6.  In  our  experiments,  we  used  different  size  chains  and  trees  and 
averaged  the  results  over  100  random  trials.  We  considered  the  best-  and  worst-case  identifier  arrangements 
for  chains  (Figure  6  left)  and  worst-case  arrangement  for  trees  (where  more  active  nodes  are  closer  to  the 
centers  of  the  trees)  with  different  branching  factors.  As  the  figure  shows,  the  average  number  of  steps 
taken  by  DBA  is  near  linear  for  the  worst-case  identifier  arrangement,  and  the  number  of  steps  is  linear  on 
trees  with  a  worst-case  identifier  arrangement  (Figure  6  right).  Furthermore,  for  a  fixed  number  of  nodes 
the  number  of  steps  decreases  inversely  when  branching  factors  of  the  trees  increase.  In  short,  DBA  is 
efficient  on  coloring  acyclic  graphs. 

Cyclic  graphs:  Unfortunately,  DBA  is  not  complete  on  cyclic  constraint  graphs.  This  will  include 
non-binary  problems  as  they  can  be  converted  to  binary  problems  with  cycles.  This  is  also  the  reason  that 
breakout  algorithm  is  not  complete  on  Boolean  satisfiability  with  three  variables  per  clause  [79],  which  is 
equivalent  to  a  constraint  with  three  variables. 

When  there  are  cycles  in  a  graph,  conflicts  may  walk  on  these  cycles  forever.  To  see  this,  consider  a 
problem  of  coloring  a  ring  with  an  even  number  of  nodes  using  two  colors  (black  and  white),  as  shown 
in  Figure  7,  where  the  node  identifiers  and  constraint  weights  are  respectively  next  to  nodes  and  edges. 
Figure  7(1)  shows  a  case  where  two  conflicts  appear  at  locations  between  nodes  1  and  3  and  between 
nodes  4  and  5,  that  are  not  adjacent  to  each  other.  The  weights  of  the  corresponding  edges  are  increased 
accordingly  in  Figure  7(2).  As  node  1  (node  4)  has  a  higher  priority  than  node  3  (node  5),  it  changes  its 
value  and  pushes  the  conflict  one  step  counter-clockwise  in  Figure  7(3).  The  rest  of  Figure  7  depicts  the 
subsequent  steps  until  all  constraint  weights  have  been  increased  to  2.  This  process  can  continue  forever 
with  the  two  conflicts  moving  in  the  same  direction  on  the  chain  at  the  same  speed,  chasing  each  other 
endlessly  and  making  DBA  incomplete. 
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Figure  7:  A  worst  case  of  DBA  for  coloring  a  ring. 
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Figure  8:  Steps  taken  by  DBA  and  variants  on  the  example  of  Figure  7  with  random  initial  assignments 
(left)  and  the  specific  assignment  of  Figure  7  (right). 

2.4.3  Stochastic  Variations 

A  lesson  that  can  be  learned  from  the  above  worst-case  scenario  is  that  conflicts  should  not  move  at  the 
same  speed.  We  thus  introduce  randomness  to  alter  the  speeds  of  possible  conflict  movements  on  cycles  of 
a  graph.  This  stochastic  feature  may  increase  DBA’s  chances  of  finding  a  solufion  possibly  wifh  a  penally 
on  convergence  lo  solufion  for  some  cases. 

DBA(wp)  and  DBA(sp):  We  can  add  randomness  lo  DBA  in  Iwo  ways.  In  Ihe  firsl,  we  use  a  proba- 
bilify  for  lie  breaking.  The  algorilhm  will  proceed  as  before,  excepl  lhal  when  Iwo  neighboring  variables 
have  Ihe  same  improvemenl  for  Ihe  nexl  step,  Ihey  will  change  Iheir  values  probabilislically.  This  means 
lhal  bolh  variables  may  change  or  nol  change,  or  jusl  one  of  Ihem.  We  call  Ihis  varialion  weak  probabilistic 
DBA,  denoted  as  DBA(wp). 

In  Ihe  second  melhod,  which  was  inspired  by  Ihe  dislribuled  slochaslic  algorilhm  [25,  27,  116],  a 
variable  will  change  if  il  has  Ihe  besl  improvemenl  among  ils  neighbors.  However,  when  il  can  improve 
bul  Ihe  improvemenl  is  nol  Ihe  besl  among  ils  neighbors,  il  will  change  based  on  a  probabilily.  This 
variation  is  more  active  lhan  DBA  and  Ihe  weak  probabilistic  variation.  We  Ihus  call  il  slrong  probabilistic 
DBA,  DBA(sp)  for  short 
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Figure  9;  DBA(wp)  and  DBA(sp)  on  grid  20  x  20  and  /c  =  4. 


One  favorable  feature  of  these  variants  is  that  no  variable  identihers  are  needed,  which  may  be  im¬ 
portant  for  some  applications  where  node  identihers  across  the  whole  network  is  expensive  to  compute. 
Moreover,  these  variants  give  two  families  of  variations  to  DBA,  depending  on  the  probabilities  used.  It 
will  be  interesting  to  see  how  they  vary  under  different  parameters,  the  topic  that  we  consider  next. 

DBA(wp)  versus  DBA(sp):  We  hrst  study  the  two  variants  on  the  example  of  coloring  an  8-node  ring 
of  Figure  7.  In  the  hrst  set  of  tests,  node  identihers  and  initial  colors  are  randomly  generated  and  10,000 
trials  are  tested.  DBA  is  unable  to  terminate  on  15%  of  the  total  trials  after  more  than  100,000  steps^,  while 
on  the  other  85%  of  the  trials  DBA  hnds  a  solution  after  5  steps  on  average  as  shown  in  Figure  8(left).  In 
contrast,  DBA(wp)  and  DBA(sp)  always  hnd  solutions  but  require  almost  twice  as  many  steps  on  average 
with  the  best  probability  around  0.6. 

In  the  second  set  of  tests,  we  use  the  exact  worst-case  initial  assignment  as  shown  in  Figure  7.  As 
expected,  DBA  failed  to  terminate.  DBA(wp)  and  DBA(sp)  hnd  all  solutions  on  1,000  trials.  Since  they 
are  stochastic,  each  trial  may  run  a  different  number  of  steps.  The  average  number  of  steps  under  different 
probability  is  shown  in  Figure  8(right). 

Next  we  study  these  two  families  of  variants  on  grids,  graphs  and  trees.  We  consider  coloring  these 
structures  using  2  colors.  For  grids,  we  consider  20  x  20,  40  x  40,  and  60  x  60  grids  with  connectivities 
equal  to  =  4  and  k  =  8.  To  simulate  inhnitely  large  grids  in  our  experiments,  we  remove  the  grid 
boundaries  by  connecting  the  nodes  on  the  top  to  those  on  the  bottom  as  well  as  the  nodes  on  the  left 
to  those  on  the  right  of  the  grids  to  create  k  =  4  grid.  For  k  —  8  grid,  we  further  link  a  node  to  four 
more  neighbors,  one  each  to  the  top  left,  top  right,  bottom  left  and  bottom  right.  This  renders  the  problem 
overconstrained  for  two-coloring.  Hence,  the  algorithms  may  only  try  to  improve  the  solution  quality  by 
minimizing  the  number  of  violated  constraints. 

The  results  of  20  x  20  grids  with  =  4  are  shown  in  Figure  9,  averaged  over  2,000  trials.  As  the 
hgures  show,  the  higher  the  probability  the  better  DBA(wp)’s  performance.  For  DBA(sp)  p  =  0.5  is  the 
best  probability. 

We  generate  2,000  graphs  with  400  nodes  with  an  average  connectivity  per  node  equal  to  A:  =  4  and 

^Our  additional  tests  also  show  that  DBA’s  failure  rate  decreases  as  the  ring  size  increases. 
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Figure  10:  DBA(wp)  (left)  and  DBA(sp)  (right)  on  graph  with  400  nodes  and  k=8. 


A;  =  8  hy  adding,  respectively,  1,600  and  3,200  edges  to  randomly  picked  pairs  of  unconnected  nodes. 
These  two  graphs  are  generated  to  make  a  correspondence  to  the  grid  structures  of  fc  =  4  and  k  =  8 
considered  previously,  except  that  both  random  graphs  are  not  two-colorahle.  All  algorithms  are  applied 
to  the  same  set  of  graphs  for  a  meaningful  comparison.  Figure  10  shows  the  results  on  graphs  with  k  =  8. 
There  is  no  significant  difference  within  the  DBA(wp)  family.  Flowever,  DBA(sp)  with  large  prohahilities 
can  significantly  degrade  to  very  poor  performance,  exhibiting  a  phenomenon  similar  to  phase  transitions. 
Since  DBA(sp)  with  high  probability  is  close  to  distributed  stochastic  algorithm  [25,  27,  116],  the  results 
here  are  in  line  with  those  of  [1 16]. 

We  also  consider  DBA(wp)  and  DBA(sp)  on  random  trees  with  various  depths  and  branching  factors. 
Due  to  space  limitations,  we  do  not  include  detailed  experimental  results  here,  but  give  a  brief  summary.  As 
expected,  they  all  find  opfimal  solufions  for  all  10,000  2-coloring  insfances.  Wifhin  DBA(wp)  family,  fhere 
is  no  significanl  difference.  However,  DBA(sp)  wifh  a  high  probability  has  a  poor  anytime  performance. 

Combining  all  the  results  on  the  constraint  structures  we  considered,  DBA(sp)  appears  to  be  a  poor 
algorithm  in  some  cases,  especially  when  its  probability  is  very  high. 

DBA(wp)  and  DBA(sp)  versus  DBA:  The  remaining  issue  is  how  DBA(wp)  and  DBA(sp)  compare 
with  DBA.  Here  we  use  the  best  parameters  for  these  two  variants  from  the  previous  tests  and  compare 
them  directly  with  DBA.  We  average  the  results  over  the  same  sets  of  problem  instances  we  used  in 
Section  2.4.3.  Figures  11,  12  and  13  show  the  experimental  results  on  grids,  random  graphs  and  trees, 
respectively.  With  their  best  parameters,  DBA(wp)  and  DBA(sp)  appear  to  be  compatible  with  DBA.  Fur¬ 
thermore,  as  discussed  earlier,  DBA(wp)  and  DBA(sp)  increase  the  probability  of  convergence  to  optimal 
solutions.  DBA(wp),  in  particular,  is  a  better  alternative  in  many  cases  if  its  probability  is  chosen  carefully. 
Stochastic  features  do  not  seem  to  impair  DBAs  anytime  performance  on  many  problem  structures  and 
help  overcome  the  problem  of  incompleteness  of  DBA  on  graphs  with  cycles. 

2.5  Comparative  Analysis  and  Application 

We  now  directly  compare  DBA  and  DSA  on  multi-coloring  problems  generated  from  the  scan  scheduling 
problem  discussed  in  Section  2.2.  In  our  experiments,  we  set  the  sensing  radius  of  a  sensor  to  one  unit,  and 
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1  10  100 
number  of  steps 

Figure  1 1 :  DBA  and  random  DBAs  on  grid  20  x  20  and  k  =  4. 


Figure  12:  DBA  and  random  DBAs  on  graph  with  400  nodes  and  k=8. 


used  a  square  of  10  x  10  units  as  the  area  to  be  monitored.  The  number  of  sensing  seetors  is  set  to  three  to 
mateh  our  hardware  system.  We  randomly  and  uniformly  plaeed  a  fixed  number  of  sensors  with  arbitrary 
orientations  in  the  square.  We  then  eonverted  these  problems  into  multi-eoloring  problems  as  deseribed 
in  Seetion  2.2.  We  experimented  with  different  values  of  maximum  allowed  eolors  T,  whieh  eorrespond 
to  the  eyele  lengths  of  seanning,  and  different  sensor  aetivation  ratios  a,  whieh  determin  how  often  the 
sensors  will  be  aetive  within  a  seanning  eyele.  In  the  following,  we  report  the  results  using  T  =  6  and 
T  =  18  with  a  =  2/3. 

For  DSA,  we  ehanged  its  probability  p  of  parallel  exeeutions  from  0.1  to  0.99,  with  an  inerement 
of  0.01.  We  used  100  instanees  for  eaeh  p.  We  evaluated  the  performanee  of  DSA  and  DBA  when 
they  have  reaehed  relatively  stable  states.  Speeitieally,  we  ran  DSA  and  DBA  to  the  point  where  their 
performanee  does  not  ehange  signifieantly  from  one  step  to  the  next.  On  all  network  sizes  we  eonsidered, 
these  algorithms’  performanee  seems  to  be  stabilized  after  256  steps.  Similar  results  have  been  observed 
after  1024,  2048  and  longer  steps.  In  the  rest  of  this  seetion,  we  report  the  results  at  256  steps. 
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Figure  13:  DBA  and  random  DBAs  on  tree  with  depth  d  =  A  and  branching  factor  k  =  A. 


Figure  14:  DSA  vs.  DBA  in  terms  of  number  of  sensors,  T  =  6. 


2.5.1  Solution  quality  in  terms  of  network  sizes 

Since  one  of  our  ultimate  objectives  is  to  choose  DBA  or  DSA  to  solve  our  distributed  scan  scheduling 
problem,  we  need  to  investigate  the  relationship  between  the  quality  of  the  schedules  found  by  these  two 
algorithms  and  the  properties  of  the  underlying  networks.  To  this  end,  we  experimentally  compared  DBA 
and  DSA  on  multi-coloring  problems  produced  from  sensor  networks  of  various  sizes.  We  changed  the 
density  of  multi-coloring  graphs  by  changing  the  number  of  sensors  N.  The  solution  quality  is  the  total 
weight  of  violated  soft  constraints  normalized  by  the  total  weight  of  soft  constraints,  measured  at  256  steps 
of  the  algorithms’  executions  when  their  performances  are  relatively  stable. 

We  run  DBA  and  DSA  with  four  different  representative  probabilities  p  of  parallel  executions,  0.1, 
0.4,  0.7  and  0.9.  We  varied  the  density  or  number  of  sensors  and  compared  the  quality  of  the  colorings 
that  DSA  and  DBA  produced.  We  changed  the  number  of  sensors  from  25  to  100  with  an  increment 
of  5  sensors,  and  from  100  to  500  with  an  increment  of  25  sensors.  We  averaged  the  results  over  100 
random  problem  instances  for  each  fixed  number  of  sensors.  Figure  14  shows  the  result  on  T  =  6.  The 
horizontal  axes  in  the  figures  are  the  numbers  of  sensors,  and  the  vertical  axes  are  the  normalized  solution 


23 


Figure  15:  DSA  vs.  DBA  in  terms  of  number  of  sensors,  T  =  18. 


quality  after  256  steps.  Longer  executions,  such  as  512  and  1024  steps,  exhibit  almost  identical  results. 
Figure  14(a)  shows  the  result  in  the  whole  range  of  25  to  500  sensors  and  Figure  14(b)  expands  the  results 
of  Figure  14(a)  in  the  range  of  25  to  100  sensors. 

As  analyzed  in  Section  2.4,  DBA  may  perform  better  than  DSA  on  underconstrained  problems,  espe¬ 
cially  acyclic  graphs.  An  underconstrained  scan  scheduling  problem  may  be  created  when  more  colors 
are  available.  Indeed,  when  we  increase  the  number  of  allowed  colors  (targeting  schedule  cycle  length)  to 
eighteen  (T  =  18),  DBA  outperforms  DSA  on  sparse  networks  with  less  than  50  sensors.  This  result  is 
shown  in  Figure  15,  where  each  data  point  is  averaged  over  100  trials. 

Based  on  the  experimental  results,  we  can  reach  three  conclusions.  First,  DBA  typically  performs 
worse  than  DSA  when  its  degree  of  parallelism  is  not  too  high  in  the  range  of  25  to  500  sensors.  Second, 
when  the  sensor  density  increases,  the  performance  of  DSA  may  degenerate,  especially  if  its  degree  of 
parallelism  p  is  high.  For  instance,  DSA  with  p  =  0.9  becomes  the  worst  of  all  when  there  are  more  than 
400  sensors  (Figure  14).  This  means  that  the  denser  the  sensor  networks  are,  the  smaller  the  parallel  degree 
p  should  be.  The  degenerated  performance  of  DSA  with  a  large  p  is  mainly  due  to  its  phase-transition 
behavior  revealed  in  the  previous  section.  When  the  sensor  density  increases,  more  constraints  will  be 
introduced  into  the  inherited  constraints  of  the  scan  scheduling  problem,  so  that  the  problem  becomes 
overconstrained.  As  indicated  in  the  phase-transition  section,  DSA’s  phase-transition  behavior  appears 
sooner  when  overall  constraints  are  tighter.  Third,  in  the  underconstrained  region,  a  higher  degree  of 
parallelism  is  preferred  to  a  lower  degree. 

2.5.2  Anytime  performance 

An  important  feature  of  our  targeting  sensor  network  for  object  detection  is  real-time  response.  High 
real-time  performance  is  important,  especially  for  systems  with  limited  computation  and  communication 
resources  in  which  it  may  be  disastrous  to  wait  for  the  systems  to  reach  stable  or  equilibrium  states.  This 
is  particularly  true  for  our  sensor-based  system  for  object  detection  and  mobile  object  tracking.  Therefore, 
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Figure  16:  Anytime  performance  of  DSA  and  DBA  in  dense  sensor  networks,  T  =  6. 


Figure  17:  Anytime  performance  of  DSA  and  DBA  in  sparse  sensor  networks,  T  =  6. 


the  algorithm  for  the  distributed  scan  scheduling  must  have  anytime  property,  i.e.,  the  algorithm  can  be 
stopped  at  anytime  during  its  execution  and  is  able  to  provide  a  feasible  solution  at  that  point.  Fortunately, 
DBA  and  DSA  can  both  be  used  for  this  purpose  because  the  hard  constraints  internal  to  individual  sensors 
(a  sensor  cannot  scan  its  two  sectors  at  the  same  time)  are  always  maintained. 

In  the  rest  of  this  section,  we  directly  compare  DBA  and  DSA  as  anytime  algorithms.  We  use  the  same 
set  of  experimental  conditions  and  parameters  as  in  the  previous  sections,  i.e.,  sensors  have  three  sectors 
and  are  randomly  and  uniformly  placed  on  a  10  x  10  grid,  with  results  averaged  over  100  trials. 

We  first  consider  dense  networks  with  N  =  500  and  N  =  300  sensors.  Based  on  the  phase-transition 
results  in  Section  2.3.2,  DSA  performs  the  best  with  p  =  0.9  and  p  =  0.78  for  the  networks  of  =  500 
and  N  =  300  nodes,  respectively.  We  used  these  parameters  in  our  experiments.  The  experimental  results 
are  in  Figure  16.  As  the  results  show,  DSA  performs  much  better  than  DBA  in  both  anytime  performance 
and  final  solution  qualify. 

We  now  consider  sparse  sensor  nefworks,  using  nefworks  wifh  N  =  80  and  =  40  sensors  as 
represenfafives.  As  discussed  earlier,  a  higher  degree  of  parallelism  should  be  used  on  sparse  graphs,  we 
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Figure  18:  Anytime  performance  of  DSA  and  DBA  in  dense  sensor  networks,  T  =  18. 


Figure  19:  Anytime  performance  of  DSA  and  DBA  in  sparse  sensor  networks,  T  =  18. 

thus  use  p  =  0.9  for  our  two  sparse  networks.  The  results,  averaged  over  100  trials,  are  in  Figure  17. 
Clearly,  DBA  and  DSA  exhibits  similar  performance,  with  DSA  being  able  to  produce  slightly  better 
solutions  at  the  end. 

To  complete  our  analysis,  we  also  compared  DSA  and  DBA  on  the  same  sets  of  instances,  but  with  18 
available  colors  (T  =  18).  The  results  on  dense  and  sparse  sensor  networks  are  included  in  Figures  18  and 
19,  respectively.  Interestingly,  DBA’s  anytime  performance  degenerates,  compared  to  that  using  T  =  6. 

In  summary,  as  far  as  solution  quality  (anytime  and  final  solutions)  is  concerned,  our  experimental 
results  indicate  that  DSA  should  be  adopted  for  the  distributed  scan  scheduling  problem. 

2.5.3  Communication  Cost 

We  have  so  far  concentrated  on  the  solution  quality  of  DBA  and  DSA  without  paying  any  attention  to  their 
communication  cost.  As  mentioned  earlier,  communication  in  a  sensor  network  has  an  inherited  delay 
and  could  be  unreliable  in  most  situations.  Therefore,  a  good  distributed  algorithm  should  require  a  small 
number  of  message  exchanges. 
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Figure  20:  Communication-cost  phase  transitions  of  DS  A  on  scan  scheduling,  T  =  6. 


In  each  step  of  DBA,  an  agent  announces  its  best  possible  conflict  reduction  to  its  neighbors  and 
receives  from  the  neighbors  their  possible  weight  reductions.  Thus,  the  number  of  messages  sent  and 
received  by  an  agent  in  each  step  of  DBA  is  no  less  than  the  number  of  its  neighbors,  and  the  total  number 
of  messages  exchanged  in  each  step  is  more  than  a  constant  for  a  given  network. 

In  contrast,  an  agent  may  not  have  to  send  a  message  in  a  step  in  DSA  if  it  does  not  change  its  value. 
In  an  extreme  case,  an  agent  will  not  change  its  value  if  it  is  at  an  local  minima.  If  solution  quality  of  DSA 
improves  over  time,  its  communication  cost  will  reduce  as  well.  In  principle,  the  communication  cost  of 
DSA  is  correlated  to  its  solution  quality.  The  better  the  current  solution,  the  less  the  number  of  messages. 
In  addition,  the  communication  cost  is  also  related  to  the  degree  of  parallel  executions  of  the  agents.  The 
higher  the  parallel  probability  p  is,  the  higher  the  communication  cost  will  be.  As  shown  in  Section  2.3, 
the  communication  cost  of  DSA  goes  hand-in-hand  with  its  solution  quality  and  also  experiences  a  similar 
phase-transition  or  threshold  behavior  on  regular  coloring  problems.  Figure  20  shows  the  phase-transition 
behavior  of  DSA’s  communication  cost  on  the  N  =  500  and  N  =  300  sensor  networks  using  T  =  6 
that  we  studied  before.  Here  we  considered  the  accumulative  communication  cost  of  all  256  steps.  This 
result  indicates  that  the  degree  of  parallelism  must  be  controlled  properly  in  order  to  make  DSA  effective. 
Similar  phase-transition  patterns  have  been  observed  when  we  use  T  =  18. 

We  now  compare  DSA  and  DBA  in  terms  of  communication  cost.  Figure  2 1  shows  the  results  eval¬ 
uating  DBA  and  DSA  with  probability  p  =  0.78  on  =  500  networks  using  T  =  6  and  T  =  18, 
averaged  over  100  trials.  The  figures  plot  the  average  numbers  of  messages  exchanged  in  DSA  and  DBA 
at  a  particular  step.  Clearly,  DSA  has  a  significant  advantage  over  DBA  on  communication  cost.  The 
large  difference  on  communication  cost  between  DSA  and  DBA  will  have  a  significant  implication  on 
how  these  two  algorithms  can  be  used  in  real  sensor  networks,  especially  when  the  sensors  are  connected 
through  delayed,  unreliable  and  noisy  wireless  communication.  For  our  particular  application  and  system 
where  communication  was  carried  out  by  radio  frequencies,  DBA’s  high  communication  cost  makes  it 
noncompetitive. 

In  summary,  in  terms  of  solution  quality  and  communication  cost,  DSA  is  preferable  over  DBA  for  our 
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Figure  21:  Communication  cost  of  DSA  and  DBA,  T  =  6  (left)  and  T  =  18  (right). 


Figure  22:  Finding  best  possible  schedule  using  DSA  (top)  and  DBA  (bottom),  N  =  80. 


distributed  scan  scheduling  if  DBA’s  degree  of  parallelism  is  properly  controlled. 

2.5.4  Solving  Scheduling  Problem 

Based  on  the  results  from  Sections  2.3.2  to  2.5.3,  we  now  apply  DSA  and  DBA  to  dealing  with  two  related 
problems  at  the  same  time,  finding  the  shortest  scan  cycle  length  T  and  obtaining  a  good  schedule  given 
the  shortest  cycle  length  T. 

To  this  end,  we  run  DSA  and  DBA  in  iterations,  starting  with  an  initially  large  T.  T  is  reduced  after 
each  iteration.  Given  a  T  in  an  iteration,  DSA  or  DBA  searches  for  a  schedule  of  a  quality  better  than  a 
predefined  threshold  Q.  The  iteration  stops  whenever  such  a  schedule  is  found  within  a  fixed  number  of 
steps,  and  a  new  iteration  may  start  with  a  smaller  T. 

In  our  simulation,  we  checked  the  quality  of  the  current  schedule  after  each  simulated  step.  As  soon 
as  the  quality  of  the  current  schedule  exceeds  the  given  threshold  Q,  we  terminate  the  current  iteration. 
This  is  equivalent  to  having  an  agent  compute  the  global  state  of  a  distributed  system,  a  method  infeasible 
for  our  completely  distributed  system.  We  use  this  mechanism  here  simply  to  evaluate  the  performance  of 
DSA  and  DBA. 
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Figure  23:  Finding  best  possible  schedule  using  DSA  (top)  and  DBA  (bottom),  N  =  300. 


Figures  22  and  23  show  the  results  on  two  networks,  one  with  =  80  sensors  and  the  other  with 
N  =  300.  In  our  experiments,  we  fixed  the  sensor  activation  ratio  at  a  =  2/3,  used  initial  T  =  36,  and 
reduced  T  by  three  after  each  iteration,  which  ran  a  maximum  of  256  steps.  The  threshold  for  schedule 
quality  was  set  to  Q  =  0.01  for  =  80  and  Q  =  40  for  A^  =  300.  As  the  results  show,  DSA  is  superior 
to  DBA.  On  the  A^  =  80  network  (Figure  22),  DSA  finds  a  fargefing  schedule  of  lengfh  T  =  15  in  242 
steps,  while  DBA  needs  588  steps.  On  fhe  N  =  300  nefwork  (Figure  23),  DSA  lakes  64  steps,  while  DBA 
uses  691  sleps,  which  is  an  order  of  magnifude  difference. 

In  summary,  our  resulfs  clearly  show  lhal  DSA  is  superior  lo  DBA  on  fhe  dislribufed  scan  scheduling 
problem.  If  communication  cost  is  also  a  concern,  DSA  is  definitely  the  algorithm  of  choice  for  the 
problem. 

2.6  Related  Work  and  Discussions 

The  basic  idea  of  distributed  stochastic  search  must  have  been  around  for  some  time.  A  similar  idea  was 
used  by  Pearl  in  distributed  belief  update  [83].  The  idea  was  directly  used  for  distributed  graph  coloring 
in  [25,  27].  DSA-B  considered  here  is  the  same  as  CFP  in  [27].  However,  [25,  27]  failed  to  reveal  phase 
transitions  discussed  in  this  section.  The  idea  was  also  studied  using  spin  glasses  models  [67]  where 
phase  transitions  were  characterized.  Phase  transitions  in  distributed  constraint  problem  solving  was  also 
reported  in  [42]. 

This  research  extends  the  existing  work  of  DSA  in  many  different  ways.  It  proposes  two  variations 
to  the  basic  DSA.  It  systematically  studies  observation-based,  distributed  stochastic  search  for  distributed 
coordination  and  provides  an  experimental,  qualitative  analysis  on  the  relationship  among  the  degree  of 
parallelism,  problem  constrainedness,  solution  quality  and  overall  system  behavior  such  as  phase  transi¬ 
tions.  It  also  demonstrates  that  phase  transitions  exist  in  many  different  problems  and  problem  structures 
and  they  persist  when  the  degree  of  parallelism  changes.  Notice  that  the  phase  transitions  considered  in 
this  section  are  different  from  phase  transitions  of  graph  coloring  problems  [14].  Here  we  studied  the 
phase-transition  behavior  of  distributed  search  algorithms,  which  needs  not  be  phase  transitions  of  the 
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coloring  problems  we  considered. 

Other  related  algorithms  include  complete  algorithms  for  DisCSP,  such  as  the  asynchronous  weak- 
commitment  (AWC)  search  algorithm  [103,  104],  and  for  DisCOP.  These  complete  algorithms  are  impor¬ 
tant  for  distributed  constraint  solving.  Comparing  to  DBA  and  DSA,  however,  they  require  longer  running 
time  and  usually  have  worse  anytime  performance,  making  them  inferior  for  real-time  applications  where 
optimal  solutions  may  be  too  costly  or  may  not  be  necessary.  In  addition,  these  complete  algorithms  need 
a  sufficiently  large  amount  of  memory  to  record  the  states  (agent  views)  that  an  agent  has  visited  in  order 
to  avoid  revisiting  a  state  multiple  times  so  as  to  make  the  algorithms  converge  to  a  solution  if  it  exist.  In 
contrast,  DSA  and  DBA  are  able  to  reach  near  optimal  solutions  quickly  without  additional  memory.  This 
feature,  along  with  their  good  anytime  performance,  made  DSA  and  DBA  attractive  applications  in  sensor 
networks  where  memory  is  a  critically  limited  resource.  In  addition,  on  coloring  acyclic  graphs,  DBA  is 
complete  and  has  a  low  polynomial  complexity,  making  it  an  alternative  algorithm  for  optimal  solution  in 
such  a  case. 

DSA  differs  from  DBA,  AWC  and  Adopt  by  the  notion  of  uniformness.  A  distributed  algorithm  is 
called  uniform  if  all  nodes  execute  the  same  procedure  and  two  nodes  do  not  differ  from  each  other  [94]. 
Therefore,  DSA  is  a  uniform  algorithm  since  the  nodes  do  not  have  identifications  and  they  all  execute 
the  same  set  of  instructions.  However,  DBA,  AWC  and  Adopt  are  not  uniform  because  the  nodes  in  these 
algorithms  need  to  have  identifications  to  differ  from  one  another  and  to  set  priorities  to  decide  what  to 
execute  next. 

We  need  to  emphasize  that  the  notion  of  uniformness  for  distributed  algorithms  has  a  practical  impor¬ 
tance  for  applications  in  sensor  networks.  In  a  typical  application  in  sensor  networks,  sensors  may  have  to 
dynamically  organize  to  form  a  system.  The  use  of  identifications  of  and  priorities  among  nodes  (sensors) 
will  introduce  prohibitive  barriers  on  what  systems  a  set  of  sensors  and  a  given  placement  can  form.  The 
fact  that  DSA  is  a  uniform  algorithm  further  supports  the  conclusion  from  the  comparison  results  in  this 
section  that  DSA  is  the  algorithm  to  choose  over  DBA. 

2.7  Conclusions 

We  were  motivated  in  this  research  to  apply  the  framework  of  multiagent  systems  and  the  techniques 
of  distributed  constraint  problem  solving  to  resource  bounded,  anytime,  distributed  constraint  problems 
in  sensor  networks.  Our  specific  applicafions  include  fhe  defecfion  of  mobile  objecfs  and  fhe  defecfion  of 
maferial  damage  in  real  lime  using  dislribuled  sensors  and  aclualors.  We  firsl  formulaled  Ihese  problems  as 
disfribuled  mulfiple  coloring  problems  wifh  fhe  objecfive  of  minimizing  fhe  number  of  violafed  consfrainls. 

To  cope  wifh  limited  resources  and  lo  meel  fhe  resfricled  requiremenl  of  anyfime  performance,  we 
were  inferesfed  in  fhose  disfribuled  algorifhms  lhal  have  low-overhead  on  memory  and  compulalion  for 
solving  dislribuled  conslrainl  oplimizalion  problems.  We  focused  particularly  on  fhe  disfribuled  slochaslic 
algorilhm  (DSA)  [25,  27,  67,  83]  and  fhe  dislribuled  breakoul  algorilhm  (DBA)  [79,  103,  105],  Iwo  exisl- 
ing  dislribuled  algorilhms  lhal  til  into  Ihe  category  of  low-overhead  dislribuled  algorifhms.  We  analyzed 
and  compared  DSA  and  DBA  on  dislribuled  multiple  coloring  problems  lhal  were  generated  from  our 
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distributed  scheduling  problems  in  sensor  networks.  We  specifically  investigated  the  relationship  among 
the  degree  of  parallel  executions,  problem  constrainedness,  and  DSA’s  behavior  and  performance.  We 
showed  that  DSA  exhibits  a  threshold  behavior  similar  to  phase  transitions  in  which  its  performance,  in 
terms  of  both  solution  quality  and  communication  cost,  degrades  abruptly  and  dramatically  when  the  de¬ 
gree  of  agents’  parallel  execution  increases  beyond  a  critical  point.  We  also  studied  the  completeness 
and  complexity  of  DBA  on  distributed  graph  coloring  problems,  showing  that  DBA  is  complete  and  has 
low  polynomial  complexity  on  coloring  acyclic  graphs.  However,  DBA  is  not  complete  in  general.  We 
also  introduced  randomization  schemes  to  DBA  to  improve  its  worst  case  performance.  Finally,  we  di¬ 
rectly  compared  DSA  and  DBA  on  our  application  problems  of  distributed  scheduling  problems  in  sensor 
networks.  We  showed  that  if  controlled  properly,  DSA  is  significantly  superior  to  DBA,  finding  beffer 
solufions  wifh  less  compufafional  cosf  and  communicafion  overhead.  For  disfribufed  scheduling  problems 
such  as  fhe  ones  considered  in  fhis  secfion,  DSA  is  fhe  algorifhm  of  choice. 

One  imporfanf  lession  we  have  learned  from  fhis  research  was  fhaf  full  syncronizafion  may  nol  be 
absolufely  necessary  in  achieving  high  degree  of  opfimalify  in  disfribufed  environmenfs.  This  is  even 
more  so  when  compufafional  and  communicafional  resources  are  crifically  limifed  and  consfrained,  which 
appear  very  oflen  in  Aufonamous  Negofiafing  Teams  domains. 
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3  Analysis  of  Negotiation  Protocols  by  Distributed  Search 


Negotiation  is  an  important  mechanism  for  coordination  and  collaboration  in  multi-agent  systems.  It 
is  particularly  effective  for  systems  consisting  of  self-interested  agents,  each  of  which  may  have  different 
objectives  to  achieve  and  different  restrictions  to  abide  to.  It  is  perhaps  also  prevalent  in  applications  where 
privacy  of  individual  agents  needs  to  be  protected.  Due  to  its  importance,  negotiation  has  been  studied  for 
quite  some  time,  and  many  different  negotiation  strategies  and  protocols  have  been  proposed  and  developed 
[11,  2,  17,  23,  24,  62].  Cooperative  negotiation  has  been  studied  to  solve  difficult  distributed  problems 
such  as  distributed  conflict  resolution  [2,  62],  distributed  task  allocation  [11,  24],  and  distributed  resource 
allocation  [17].  In  all  these  domains,  multiple  agents  share  some  common  resources,  e.g.,  communication 
channels  and  CPU  times,  and  can  mutually  benefit  from  one  another  by  cooperatively  scheduling  the 
resources.  For  these  problems,  agents  only  have  the  information  about  their  local  tasks  and  resources.  They 
gradually  become  aware  of  the  global  information  through  negotiation  and  solve  the  global  problem  by 
individually  solving  sub-problems  and  integrating  the  solutions  to  sub-problems  into  a  globally  consistent 
solution  [64].  For  some  applications,  agents  may  also  need  to  assess  and  refine  fhe  global  solufions  info 
global  opfimal  solufions  fo  make  fhe  besf  use  of  common  resources. 

However,  global  opfimal  solufions  are  very  hard  fo  achieve  fhrough  cooperafive  negofiafion  in  a  dis- 
fribufed  seffing,  due  fo  ifs  compufafional  and  communicafional  complexify.  Furfhermore,  mosf  exisfing 
negofiafion  profocols  are  complex  and  fheir  feafures  are  difficulf  fo  characferize.  To  our  knowledge,  we 
have  nof  seen  a  published  work  fhaf  analyzes  a  negofiafion  profocol  in  depfh  fo  undersfand  imporfanf  issues 
of  a  negofiafion  mefhod,  such  as  ifs  complefeness,  complexify  and  scalabilify. 

In  fhis  section,  we  propose  an  experimenfal  approach  fo  analyzing  negofiafion  mefhods.  Our  sfrafegy 
consisfs  of  fwo  sfeps.  In  fhe  firsf  sfep,  we  formulafe  fhe  disfribufed  problems  solved  by  a  negofiafion 
mefhod  by  disfribufed  consfrainf  safisfacfion/opfimizafion  problems,  and  capfure  fhe  negofiafion  profocol 
as  a  disfribufed  consfrainf  search.  In  fhe  second  sfep,  we  sfudy  fhe  properfies  of  fhe  negofiafion  profocol 
by  analyzing  fhe  derived  search  algorifhm.  Nofe  fhaf  fo  a  larger  exfenf,  fhe  idea  of  viewing  negofiafion  as 
disfribufed  search  is  nof  complefely  new.  Indeed,  if  has  been  suggesfed  fhaf  disfribufed  AI  can  be  viewed  as 
disfribufed  search  [64] .  Neverfheless,  we  nof  only  view  negofiafion  as  disfribufed  search  in  fhis  research, 
buf  fake  fhis  view  one  sfep  furfher  and  direcfly  apply  a  search  algorifhm  fo  capfure  fhe  essenfial  feafures  of 
a  negofiafion  profocol  and  analyze  ifs  properfies  and  performance. 

In  fhis  research,  we  specifically  focus  on  a  resource  allocafion  problem  underlying  a  mobile  objecf 
fracking  problem  using  disfribufed  sensors  and  a  recenfly  developed  negofiafion  profocol.  Scalable  Profo¬ 
col  for  Anytime  Multi-level  negofiafion  (SPAM)  [68],  for  fhis  problem.  The  SPAM  negofiafion  profocol 
has  been  successfully  used  fo  manage  a  sef  of  disfribufed  sensors  fo  solve  fhe  problem  of  fracking  mulfiple 
fargefs.  Specifically,  a  sef  of  nef worked  sensors  cooperafively  defecf  and  localize  a  sef  of  moving  fargefs 
by  faking  local  measuremenfs  and  exchanging  informafion.  Since  a  sensor  is  only  capable  of  measuring 
fhe  disfance  from  a  fargef  fo  fhe  sensor  as  well  as  fhe  speed  of  fhe  fargef,  in  order  fo  esfimafe  fhe  location  of 
fhe  fargef,  mulfiple  sensors  have  fo  defecf  af  fhe  same  fime  and  combine  fheir  measuremenfs.  Having  mea¬ 
suremenfs  from  more  sensors  af  fhe  same  fime  or  geffing  fhe  measuremenfs  more  frequenfly  will  produce 
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a  higher  quality  tracking. 

The  primary  objective  of  this  multiple  target  tracking  problem  is  to  allocate  the  sensors  to  the  targets 
so  as  to  maximize  the  tracking  quality.  The  SPAM  protocol  is  designed  for  solving  this  problem  with 
distributed  negotiating  agents.  Each  sensor  is  associated  and  managed  by  an  agent.  Whenever  a  new 
target  is  detected,  an  agent  may  also  play  the  role  of  managing  the  task  of  keeping  track  of  the  target,  i.e., 
determining  which  sensors  to  use  for  tracking  and  making  schedules  for  the  available  sensors.  Such  an 
agent  is  also  called  a  track  manager  in  the  protocol.  When  more  than  one  target  enters  the  system,  conflicts 
on  the  demand  of  sensors  exist,  and  track  managers  may  need  to  negotiate  with  one  another  to  resolve  the 
conflicts  on  their  local  schedules  and/or  cooperatively  produce  a  global  schedule  to  attempt  to  maximize 
the  overall  tracking  quality.  The  key  idea  of  the  SPAM  protocol  is  to  select  one  of  the  conflicting  track 
managers  as  a  mediator  to  resolve  the  conflicts  and  generate  partial  solutions  for  the  conflicting  managers 
involved.  This  conflict  resolution  process  may  propagate  to  multiple  negotiation  threads.  An  experimental 
study  showed  that  the  SPAM  protocol  works  very  well  for  this  real-time  sensor  tracking  problem.  However, 
the  SPAM  protocol  is  too  complex  to  be  amenable  to  a  thorough  theoretical  analysis,  and  almost  all  of  its 
important  features,  such  as  completeness  and  convergency,  have  not  been  analyzed. 

In  addition  to  developing  a  general  experimental  approach  of  analyzing  negotiation  strategies,  another 
objective  of  this  research  is  to  elucidate  the  properties  of  SPAM,  including  its  completeness,  convergence, 
complexity  and  robustness.  To  this  end,  we  view  the  resource  (sensor)  allocation  problem  for  target  track¬ 
ing  as  a  distributed  constraint  satisfaction/optimization  problem  and  transform  the  SPAM  protocol  into 
distributed  search  algorithms.  We  then  characterize  many  important  features  of  the  SPAM  protocol  by 
analyzing  the  derived  search  algorithms. 

This  section  is  organized  as  follows.  We  briefly  describe  the  sensor  tracking  problem  and  the  SPAM 
negotiation  protocol  in  section  3.1.  We  formulate  the  cooperative  negotiation  problem  as  a  multi-agent 
constraint  problem  in  section  3.2.  In  section  3.3,  we  describe  our  strategy  to  characterizing  the  negoti¬ 
ation  as  distributed  constraint  search,  and  further  generalizing  and  extending  the  SPAM  protocol  as  two 
distributed  constraint  search  algorithms.  In  section  3.4  we  apply  the  derived  search  algorithms  to  experi¬ 
mentally  analyze  some  essential  features  of  the  negotiation  protocol.  Finally  we  conclude  and  summarize 
our  results  in  section  3.5. 

3.1  Target  Tracking  and  the  SPAM  Protocol 

The  application  we  consider  is  the  tracking  of  multiple  moving  targets  using  a  network  of  loosely  coupled 
sensors.  This  application  has  also  motivated  the  SPAM  negotiation  protocol.  We  will  briefly  describe  the 
problem  in  section  3.1.1  and  the  protocol  in  section  3.1.2. 

3.1.1  Tracking  multiple  targets 

In  this  problem,  a  set  of  Doppler  sensors  are  scattered  with  varying  orientations  throughout  a  tracking 
area.  Each  sensor  is  able  to  detect  an  object  within  a  fixed  radius.  However,  the  overall  detection  area  of  a 
sensor  is  divided  into  three  equal  sectors,  and  the  sensor  can  only  operate  in  one  sector  at  any  given  time. 
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The  Doppler  sensor  is  only  eapable  of  deteeting  the  distanee  between  a  target  and  the  sensor  by  measuring 
signal  amplitude  as  well  as  the  speed  of  the  target  based  on  signal  frequeney.  In  order  to  traek  a  target,  at 
least  three  sensors  are  required  to  sense  at  the  same  time  and  triangulate  the  loeation  of  the  target.  Having 
more  sensors  traeking  at  the  same  time  or  taking  measurements  more  frequently  will  produee  a  higher 
quality  traeking. 

The  problem  ean  thus  be  deseribed  as  follows.  Given  n  sensors  and  m  targets,  develop  a  sehedule 
for  all  the  sensors  to  traek  the  maximal  number  of  targets  as  aeeurately  and  frequently  as  possible  over 
a  period  of  time.  The  eomplieation  of  the  problem  stems  in  at  least  two  faetors.  First,  multiple  targets 
ean  exist  and  eonfiiets  on  alloeating  a  sensor  to  multiple  targets  may  be  unavoidable,  espeeially  when  the 
number  of  sensors  is  not  suffieient.  Seeond,  an  agent  has  partial  knowledge  of  the  overall  problem  but 
needs  to  aet  using  loeal  information  and  information  from  its  neighboring  agents. 

From  utility  theoretieal  point  of  view,  the  problem  ean  be  eonsidered  as  a  problem  of  maximizing  a 
utility  funetion  of  the  targets  being  traeked  over  a  period  of  time.  If  the  number  of  sensors  alloeated  to 
a  partieular  target  is  less  than  three,  no  reliable  estimation  to  the  loeation  of  the  target  ean  be  obtained. 
On  the  other  hand,  even  though  having  more  than  three  sensors  alloeated  to  a  target  will  give  rise  to  better 
traeking  result,  the  traeking  quality  is  not  linear  of  the  number  of  sensors  used  and  the  quality  improvement 
will  be  negligibly  small  after  a  speeifie  number  of  sensors.  For  example,  the  utility  funetion  of  traeking 
a  set  of  m  targets  ean  be  defined  as  the  total  sum  of  the  utility  of  traeking  individual  targets,  eaeh  of 
whieh  ean  be  defined  as  s  x  ^  y/3l  —  5  if  I  sensors  are  alloeafed  fo  a  fargef  during  a  period  fime  s.  This 
ufilify  funetion  eould  be  more  eomplieafed  if  if  fakes  info  aeeounf  fhe  loeation  and  fhe  orienfafion  of  eaeh 
individual  sensor  wifh  respeef  fo  fhe  large!  being  Iraeked. 

3.1.2  The  SPAM  protocol 

The  SPAM  profoeol  is  designed  wifh  many  eonsiderafions  on  real-time  performanee  and  dynamie  issues. 
The  profoeol  is  divided  info  fhree  sfages  or  absfraelion  levels  fo  eope  wifh  fhree  levels  of  real-time  eon- 
sfrainfs.  In  fhe  lowesf  or  sensor  level,  fhe  syslem  is  required  fo  respond  immedialely  so  lhal  fhe  problem 
is  solved  wifhoul  any  informalion  on  eaeh  loeal  sensor.  Af  fhe  end  of  fhis  sfage,  many  eonfliels  over  fhe 
alloealion  of  fhe  sensors  may  exisf.  Al  fhe  seeond  absfraelion  level,  fhe  problem  is  solved  wifh  knowing 
fhe  sehedule  on  eaeh  loeal  sensor.  Al  fhe  end  of  fhis  sfage,  all  fhe  loeal  eonfliels  ean  be  solved,  bul  some 
non-loeal  eonfliels  may  be  inlrodueed  due  fo  missing  fhe  global  information.  Al  fhe  Ihird  or  fhe  resouree 
level,  fhe  eonflieling  agenls  negoliale  over  Iheir  loeal  sehedules  Ihrough  a  medialor  who  generales  parlial 
solulions  for  all  Ihese  agenls  and  fries  fo  solve  non-loeal  eonfliels.  As  long  as  eonfliels  exisf  in  fhe  syslem, 
some  agenl  will  beeome  a  medialor  and  propagale  fhe  negolialion.  A  brief  deseriplion  of  Ihese  fhree  sfages 
are  given  below. 

Bofh  simulation  and  hardware  experimenls  show  lhal  fhe  SPAM  profoeol  works  very  well  for  fhe  real¬ 
time  moving  large!  Iraeking  problem.  However,  sinee  fhe  profoeol  ilself  is  loo  eomplex  fo  analyze,  some 
imporlanl  fealures  of  fhe  profoeol  are  still  unelear.  We  are  espeeially  interested  in  fhe  following  properlies 
of  fhe  profoeol:  eompleleness,  lime  eomplexily,  rate  of  eonvergenee,  and  sealabilily.  In  order  fo  eaplure 
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Algorithm  3  SPAM  Protocol  Stage  0 
Evaluate  and  order  the  usable  resources 
Decides  an  initial  objective  level 
if  (have  more  time)  then 
Go  to  Stage  1 
else 

Choose  a  solution  maximizing  the  local  utility 
Bind  the  solution  and  exit 

end  if 


Algorithm  4  SPAM  Protocol  Stage  1 

Collect  the  local  information  from  all  usable  resources 
Generate  the  set  of  local  solutions 
if  (have  solution  without  conflicts)  then 

Choose  the  solution  with  the  maximum  local  utility 
Bind  the  solution 
else 

Choose  a  solution  which  both  minimizes  the  conflicts  and  maximizes  the  local  utility 
Bind  the  solution  and  exit 
if  (have  more  time)  then 
Go  to  Stage  2 

end  if 
end  if 


these  properties,  we  will  transform  the  protocol  into  succinct  distributed  constraint  search  algorithms.  By 
analyzing  these  search  algorithms,  we  can  characterize  many  important  features  of  the  protocol.  Before 
we  launch  onto  search  algorithms,  we  first  describe  the  constraint  formulation  of  the  tracking  problem,  the 
topic  of  the  next  section. 

3.2  Constraint  Problems  in  Cooperative  Negotiation 

A  distributed  problem  that  can  be  solved  by  cooperative  negotiation  normally  involves  a  set  of  agents, 
each  of  which  has  some  tasks  to  be  scheduled  using  a  set  of  shared  resources.  Given  a  set  of  n  agents 
over  m  resources,  a  cooperative  negotiation  problem  can  be  formally  represented  as  (A,  R),  where  A  = 
{oi,  02,  •  •  •  ,  ttn}  is  the  set  of  n  agents  and  R  =  {si,  S2,  •  •  •  ,  Sm}  the  set  of  m  resources.  An  agent,  ai,  is 
represented  by  a  tuple: 

Ci,  Wi) , 

where  Ri,  Ri  C  i?,  is  a  set  of  resources  that  can  be  used  by  agent  a*,  Ci  represents  a  task  to  be  scheduled 
by  tti,  and  Wi  is  the  weight  assigned  to  task  C*. 

The  task  of  Ci  may  also  consists  of  a  set  of  sub-tasks  T,  =  •  •  •  ,  and  {Ti,Ri) 

constitutes  a  local  sub-problem  that  can  be  internally  solved  by  agent  ai.  In  other  words,  (Tj,  Ri)  is  local 
to  ai  and  is  unknown  to  the  other  agents.  The  overall  task  Ci  requires  a  certain  number  of  resources,  and 
this  number,  represented  by  Oi,  is  called  the  objective  level.  Given  the  set  of  usable  resources  Ri  and  the 
objective  level  Oi  of  the  agent  ai,  there  are  k  =  alternative  solutions.  Then  task  Ci  can  be  represented 
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Algorithm  5  SPAM  Protocol  Stage  2 

Mediator  detects  the  oscillation  by  checking  the  history  of  negotiation 
if  (have  oscillation)  then 
Lower  the  objective  level 

end  if 

Request  meta-level  information  from  the  conflicting  agents 
while  (have  no  solution)  do 

Generate  partial  solutions  for  all  these  agents 
if  (all  agents  are  at  their  lowest  objective  level)  then 

Choose  a  solution  at  the  lowest  objective  level  with  min-conflicts 
Exit 
else 

Lower  the  objective  level  of  one  agent 

end  if 
end  while 

Mediator  sends  all  the  partial  solutions  to  the  other  agents 
The  other  agents  evaluate  and  rank  the  solutions 
Mediator  chooses  a  consistent  solution  according  to  the  feedback 
All  the  agents  bind  their  solutions 

Some  agent  propagates  the  negotiation  if  having  conflicts 


by 


c^=\Js,, 


i=i 


where  Sij  represents  one  possible  solution  for  the  agent  a*  to  the  task  Ci.  And  Si  =  •  •  ,Sil^} 

constitutes  the  solution  space  for  the  sub-problem  Ci  on  agent  ai.  Given  the  objective  level  Oj,  Sij  can  be 
further  represented  by 


Sij  =  /\irm  =  i) 

m=l 

where  Cm  ^  Ri-  Here  Cm  =  i  means  that  the  resource  Cm  is  allocated  to  task  Ci.  Moreover,  in  order  to 

Oi 

make  Ci  true,  at  lease  one  solution  soli  -  A  (^m  =  *)’  ^  Si,  must  be  true;  therefore,  all  the  resource 

m=l 

rl^  should  be  allocated  to  Ci.  We  call  soli  the  local  sub-solution  for  the  sub-problem  Ci  on  agent  a,. 

Here,  an  agent  is  a  track  manager,  and  a  resource  corresponds  to  a  sensor  in  the  SPAM  protocol. 
With  the  restriction  that  a  resource  can  only  be  used  by  one  task  at  any  time,  each  resource  should  have  a 
consistent  allocation  or  assignment  in  the  local  sub-solution  of  each  task  Ci,  which  constitutes  a  constraint 
among  tasks.  Note  that  each  task  Ci  is  distributed  among  a  set  of  agents.  Therefore,  we  can  formulate 
the  cooperative  negotiation  problem  as  the  following  distributed  constraint  satisfaction  problem:  Given  a 

n 

constraint  problem  {A,  R),  is  there  an  assignment  of  resources  R~^  C  R  such  that  f\  Ci  is  satisfied? 

i=l 

As  discussed  in  Section  3.1.1,  an  agent  a*  can  have  a  utility  function  Ui  :  Si  that  can  be  used  to 
discriminate  alternative  local  solutions.  Furthermore,  the  overall  goal  of  the  negotiation  problem  is  to  find 
a  globally  consistent  solution  with  maximal  global  utility  among  all  the  solutions.  Therefore,  since  Ci  has 
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a  weight  Wi,  the  global  utility  ean  be  simply  defined  as  Utiltotai  =  '  Utili,  where  Utili  =  Ui{soli) 

i=l 

is  the  utility  of  the  agent  a,  with  the  loeal  sub-solution  soli.  With  the  extension,  the  agent  a,  is  represented 
by  {Ri,  Ci,  Wi,  Ui),  and  we  ean  formulate  the  eooperative  negotiation  problem  as  the  following  eonstraint 
optimization  problem:  Given  a  constraint  problem  {A,  R),  what  is  the  assignment  of  resources  to  agents 

n 

such  that  the  constraint  /\  Ci  is  true  and  the  utility  function  Utotai  R  maximized? 
i=l 

3.3  Negotiation  Protocol  as  Search  Algorithms 

The  problems  solved  by  eooperative  negotiation  ean  be  formulated  as  distributed  satisfaetion  or  optimiza¬ 
tion  problems.  A  negotiation  protoeol  ean  be  viewed  as  a  distributed  seareh  proeess.  In  this  seetion,  we 
propose  to  eharaeterize  the  negotiation  protoeol  by  a  distributed  seareh  algorithm. 

3.3.1  Negotiation  as  distributed  search 

Negotiation  and  seareh  are  fundamentally  different.  The  former  is  naturally  a  multi-agent  problem  solving 
method  in  whieh  information  may  not  be  shared  among  agents.  Thus,  negotiation  is  generally  harder  than 
seareh,  sinee  a  global  view  of  a  problem  may  never  be  eonstrueted.  In  sueh  a  situation,  it  may  beeome 
diffieult  for  an  agent  to  even  determine  whether  the  eurrent  variable  assignments  are  in  a  better  state  than 
the  previous  ones.  Moreover,  a  negotiation  proeess  may  be  trapped  in  an  infinite  negotiation  loop  in  whieh 
all  the  agents  revisit  some  previously  eneountered  global  solutions  endlessly.  Furthermore,  negotiation 
may  have  more  restrietions  than  seareh.  For  example,  time  or  some  other  parameters  ean  beeome  a  faetor 
when  evaluating  the  results  of  the  negotiation.  Thus,  the  negotiation  is  normally  more  eomplieated  in  the 
sense  of  searehing  for  a  solution. 

Although  negotiation  and  seareh  are  different  approaehes  to  problem  solving,  they  both  seareh  for 
assignments  to  variables  of  a  problem  whieh  eonstitute  solutions.  Seareh  ean  also  be  an  ingredient  of 
negotiation.  In  a  negotiation  problem,  a  task  of  an  agent  ean  be  viewed  as  a  variable.  It  ean  be  assigned 
any  values  (sub-solutions)  in  its  solution  spaee.  The  sub-solutions  of  different  variables  may  eonfliet  with 
one  another  due  to  the  inter-agent  eonstraints.  When  an  agent  assigns  a  value  to  its  task,  it  also  needs 
to  make  it  eonsistent  with  the  assignments  of  other  agents  whieh  is  similar  to  what  a  normal  eonstraint 
seareh  algorithm  does.  An  agent’s  assignment,  when  communicated  to  other  agents  as  a  proposal  in  the 
negotiation,  can  be  rejected  by  the  other  agents.  Other  agents  may  provide  counter-proposals.  Thus  a 
negotiation  is  just  a  search  process,  in  which  the  agents  try  to  assign  values  or  revise  values  to  their  tasks 
to  satisfy  all  the  inter-agent  constraints. 

Generally,  a  negotiation  may  have  two  primary  goals,  to  search  for  consistent  solutions  and  to  search 
for  a  consistent  solution  of  a  maximal  utility.  The  first  goal  focuses  on  the  conflict  resolution,  which  is 
a  problem  solved  routinely  by  constraint  satisfaction  search  algorithms.  In  this  regard,  negotiation  can 
thus  be  viewed  as  a  constraint  satisfaction  method.  The  second  goal  of  finding  a  consisfent  solution  of  a 
maximum  utility  is  simply  an  optimization  problem,  a  harder  search  problem.  Note  that  a  task  of  an  agent 
can  be  a  complicated  subproblem,  and  the  solution  space  of  the  subproblem  itself  may  be  large.  All  agents 
need  to  search,  cooperatively,  to  find  the  best  possible  global  solutions. 
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In  short,  negotiation  can  be  viewed  as  a  mechanism  for  solving  distributed  constraint  satisfaction 
and  constraint  optimization  problem.  Taking  this  view,  we  propose  to  use  search  algorithms  as  tools  for 
analyzing  negotiation  protocols. 

3.3.2  SPAM  protocol  as  search  algorithms 

The  original  SPAM  protocol  [68]  has  many  features  to  handle  real-time  and  dynamics  issues.  Although 
these  features  are  very  necessary  and  important  to  deal  with  real-world  applications,  they  are  hardly 
amenable  to  a  thorough  analysis.  It  seems  to  be  very  difficult  to  design  an  abstract  model  of  such  a  com¬ 
plicated  protocol  for  a  theoretical  analysis.  Therefore,  it  is  difficult  to  understand,  through  an  analytical 
approach,  some  primary  properties  of  the  protocol,  such  as  completeness,  rate  of  convergence,  complexity 
and  scalability. 

The  difficulty  for  a  theoretical  analysis  suggests  that  an  experimental  analysis  is  in  demand.  Here,  we 
propose  to  use  search  algorithms  to  capture  a  negotiation  protocol  so  as  to  characterize  the  important  fea¬ 
tures  of  the  protocol  through  analyzing  the  search  algorithms.  Once  a  negotiation  protocol  is  transformed 
to  search  algorithms,  whenever  a  theoretical  analysis  is  possible  for  the  search  algorithms,  such  an  analysis 
can  also  be  translated  back  to  the  original  negotiation  protocol. 

Another  difficulty  for  analyzing  a  distributed  negotiation  protocol  comes  from  the  distributed  nature 
of  the  applications  to  which  the  protocol  is  applied.  First,  it  is  usually  difficult  to  set  up  distributed  ex¬ 
periments  with  a  large  number  of  agents  and  resources  using  a  sufficient  number  of  hardware,  while  still 
being  able  to  collect  enough  accurate  experimental  data  for  an  evaluation.  Second,  most  experiments  in 
a  distributed  environment  are  not  repeatable.  A  distributed  negotiation  protocol  is  in  essence  nondeter- 
ministic.  There  are  indeed  many  factors,  for  instance  the  synchronization  among  agents,  that  can  change 
experimental  results  from  one  run  to  another. 

Therefore,  caution  must  be  taken  in  the  experimental  analysis.  In  this  section,  we  will  first  transform 
the  SPAM  protocol  to  a  sequential  search  algorithm,  called  sequential  SPAM.  Here,  sequential  search  does 
not  necessarily  mean  centralized  search.  The  search  process  in  the  sequential  SPAM  can  still  be  distributed 
among  different  agents,  but  in  each  step  only  one  agent,  chosen  arbitrarily,  is  allowed  to  change  its  local 
values.  One  run  of  such  a  sequential  algorithm  corresponds  to  one  possible  execution  of  the  original 
negotiation  protocol.  Introducing  sequential  execution  is  expected  to  have  little  impact  on  the  effects  of 
the  protocol  while  making  the  analysis  easier.  The  sequential  search  algorithm  is  designed  to  represent 
the  features  of  the  original  protocol  as  close  as  possible,  so  that  the  results  from  the  search  algorithm  is 
expected  to  shed  some  light  on  the  original  protocol. 

We  will  then  modify  the  SPAM  protocol  to  construct  a  synchronous  search  algorithm,  called  syn¬ 
chronous  SPAM.  In  this  version,  the  agents  negotiate  in  a  more  tightly  cooperative  manner.  The  agents 
are  dynamically  ordered  during  the  negotiation.  The  agent  that  revises  its  local  assignment  earlier  has  a 
higher  priority.  This  modification  is  introduced  to  make  the  protocol  complete,  since  the  original  protocol 
cannot  guarantee  the  completeness  of  a  negotiation  process,  as  we  will  see  in  the  next  section.  Note  that 
completeness  is  not  the  same  as  optimization.  If  a  constraint  problem  is  not  satisfiable,  the  synchronous 
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STAGE  0 


Figure  24:  Sequential  SPAM 

SPAM  algorithm  is  expected  to  give  an  answer  of  ‘NO’.  Using  synchronous  SPAM,  we  are  able  to  evalu¬ 
ate  the  completeness  of  the  original  protocol.  Moreover,  we  also  expect  to  understand  how  to  improve  the 
original  protocol  by  analyzing  the  synchronous  search  algorithm. 

Sequential  SPAM 

We  now  characterize  the  SPAM  protocol  as  a  sequential  search  algorithm  in  which  only  one  agent  is 
allowed  to  change  its  local  values.  The  protocol  is  viewed  as  the  following  search  algorithm,  shown  in 
Figure  24. 

Similar  to  the  SPAM  protocol,  this  algorithm  is  divided  into  three  stages.  In  stage  0,  each  agent 
chooses  a  solution  for  its  local  problem  without  knowing  any  information  about  the  current  schedule  of 
each  resource.  In  stage  1,  each  agent  collects  the  information  of  all  the  resources  that  can  be  used.  Then 
one  of  these  agents  with  conflicts  with  other  agents,  chosen  randomly,  tries  to  resolve  the  conflicts  locally 
by  searching  for  a  solution  in  its  local  solution  space.  If  there  exists  a  local  solution  that  does  not  conflict 
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with  all  the  other  agents,  the  agent  will  ehange  to  sueh  a  non-eonfliet  solution  with  the  best  utility. 

If  there  exists  no  non-eonfliet  solution,  the  agent  will  ehoose  a  solution  whieh  minimizes  the  eonfliets 
with  the  other  agents  and  moves  on  to  stage  2.  In  other  words,  this  agent  beeomes  a  mediator  to  resolve  the 
eonfliets  in  whieh  it  is  involved.  In  stage  2,  the  agent  requests  meta-level  information  from  the  eonflicting 
agents.  It  then  tries  to  generate  solutions  for  these  agents.  If  there  exist  non-eonfliet  solutions,  these 
agents  will  ehange  to  non-eonfliet  solutions  based  on  the  heuristie  that  the  most  eonstrained  agent  ehooses 
a  solution  first. 

If  there  is  still  a  eonfliet  among  agents,  a  neighboring  eonflieting  agent  will  attempt  to  resolve  the 
eonfliet  by  propagating  the  negotiation.  To  avoid  an  infinite  negotiation  loop,  eaeh  agent  reeords  the 
history  of  its  previous  negotiations.  If  an  oseillation  is  deteeted,  the  algorithm  terminates  or  the  agent’s 
objeetive  level  is  redueed  (meaning  that  more  resourees  to  be  used)  to  lower  the  eonstrainedness  of  the 
eonfliets.  However,  as  we  will  see  later,  this  termination  eondition  will  make  the  algorithm  lose  possible 
solutions  or  solutions  with  higher  objeetive  levels.  Therefore,  the  eompleteness  of  the  algorithm  or  the 
protoeol  ean  not  be  guaranteed. 

Synehronous  SPAM 

The  synehronous  SPAM  seareh  algorithm  is  not  exaetly  the  same  as  the  original  SPAM  protoeol.  It 
simulates  the  original  protoeol  as  elose  as  possible  and  ensures  eompleteness.  The  algorithm  is  shown  in 
Figure  25. 

In  the  synehronous  SPAM  algorithm  agents  negotiate  in  a  more  tightly  eooperative  manner.  The  al¬ 
gorithm  has  additional  features,  sueh  as  priority  and  eommitment,  to  guarantee  the  eompleteness.  In  this 
algorithm,  eaeh  agent  has  a  tentative  initial  solution  for  its  loeal  problem.  The  tentative  solution  is  revised 
when  the  agent  eommits  its  solution.  All  the  eommitted  sub-solutions  eonstitute  a  partial  solution  to  the 
overall  problem.  The  revised  solutions  must  satisfy  all  the  eonstraints  with  the  sub-solutions.  If  there  exist 
multiple  solutions,  a  min-eonfliet  heuristie  [76]  is  used  to  minimize  as  many  eonfliets  with  tentative  solu¬ 
tions  as  possible.  If  no  solution  exists,  the  partial  solutions  will  be  added  as  a  new  eonstraint,  and  all  the 
eommitted  solutions  will  be  uncommitted  and  become  tentative  solutions  again.  Here  the  idea  is  borrowed 
from  weak-commitment  search  algorithm  [106]  which  has  proved  to  be  more  efficient  than  backtracking 
algorithms  in  many  cases.  After  the  partial  solution  is  uncommitted,  the  agent  will  request  meta-level 
information  from  the  other  conflicting  agents  and  try  to  generate  solutions  for  all  of  these  agents.  If  no 
solution  exists,  the  whole  problem  will  have  no  solution  at  the  current  objective  level.  As  a  result,  some 
agents  must  lower  their  objective  levels,  i.e.,  reduce  some  constraints  of  their  local  problems.  Otherwise, 
these  agents  will  just  commit  the  solutions.  Note  that  when  there  is  no  solution,  the  algorithm  will  not  try 
to  minimize  the  number  of  violated  constraints.  Thus  this  algorithm  is  for  solving  constraint  satisfaction 
not  optimization. 

As  we  will  see  in  the  next  section,  this  algorithm  can  guarantee  completeness.  It  can  also  be  easily 
modified  into  an  asynchronous  version  in  which  multiple  agents  can  search  in  parallel  but  still  guarantee 
the  completeness. 
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if  no  conflcits 
among  these  agents 


YES 


done 


3.4  Experimental  Analysis  and  Results 

To  reveal  the  properties  of  the  SPAM  protoeol,  we  now  analyze  the  eompleteness,  eonvergenee  rate,  eom- 
plexity  and  sealability  of  sequential  and  synehronous  SPAM  algorithms.  Here,  the  eompleteness  of  a 
protoeol  is  the  eapability  of  finding  a  solution  if  one  exists;  the  eonvergenee  rate  eoneerns  the  number  of 
negotiation  steps  required  to  reaeh  a  solution;  the  eomplexity  measures  the  total  number  of  steps  taken 
before  a  protoeol  terminates;  and  the  sealability  eonsiders  how  the  properties  of  a  protoeol  ehange  as  the 
size  of  a  system  inereases. 

3.4.1  Completeness 

The  synehronous  SPAM  protoeol  ean  be  proved  eomplete.  Sinee  the  algorithm  reeords  the  abandoned 
partial  solutions  as  new  eonstraints,  the  algorithm  will  not  ereate  the  same  partial  solution  twiee.  Therefore, 
the  eompleteness  of  the  protoeol  is  guaranteed,  beeause  there  are  a  finite  number  of  partial  solutions  to  be 
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enumerated.  The  worst-case  time  complexity  of  this  protocol  is  obviously  exponential  in  the  number  of 
agents.  Assuming  that  there  are  n  agents,  and  each  agent  has  a  solution  space  of  size  S,  the  worse  case 
time  complexity  will  be  Since  the  problem  itself  is  NP-complete,  this  result  seems  inevitable.  The 

worst  case  space  complexity  of  this  protocol  is  also  exponential  in  the  number  of  agents  since  if  there  is  no 
solution  for  the  whole  problem,  all  the  partial  solutions  will  be  added  as  new  constraints.  However,  unlike 
most  of  the  tree  search  algorithms,  the  synchronous  SPAM  changes  the  search  order  flexibly,  which  makes 
it  more  efficient  since  it  avoids  exhaustively  searching  all  the  bad  solutions  when  previous  values  are  set 
wrong. 
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Figure  26:  An  example  for  incompleteness 

The  sequential  SPAM  protocol  is  not  complete.  This  is  simply  because  that  the  agents  in  this  algorithm 
do  not  follow  any  ordering  of  assignments  during  the  search.  This  will  cause  some  agents  to  cycle  through 
some  sub-solutions  and  fall  into  an  infinite  search  or  negotiation  loop.  In  principle,  the  algorithm  is  similar 
to  a  local  search  and  can  step  on  a  loop  in  search  space  or  be  trapped  by  local  minima.  To  avoid  such  an 
infinite  loop  or  local  minimum,  the  protocol  simply  gives  up  the  current  search  by  lowering  the  current 
objective  level  if  an  oscillation  is  detected.  This  makes  the  algorithm  incomplete  to  find  some  possible 
solutions. 

Figure  26  gives  a  simple  example  of  incompleteness  where  four  agents  (track  managers)  try  to  uti- 
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lize  eight  resources  (sensors)  for  target  tracking.  The  problem  has  a  solution  which  gives  each  agent  a 
maximum  objective  level  of  two.  The  synchronous  SPAM  algorithm  can  find  the  solution  after  several 
steps  given  the  initial  assignment  shown  in  the  Figure  26,  while  the  sequential  SPAM  fails  to  find  such  a 
solufion,  due  fo  oscillafions  in  search  processes. 
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Figure  27:  Execution  of  sequential  SPAM  on  the  example 

Figure  27  illustrates  an  example  of  execution  steps  of  sequential  SPAM.  For  the  initial  assignments 
(after  stage  0),  ai  and  02  have  conflicts  on  the  resource  si.  ai  searches  locally  but  finds  that  there  is  no 
local  solution  to  resolve  the  conflicts  (after  stage  1).  It  then  goes  into  stage  2  to  negotiate  with  02.  It 
finally  finds  a  partial  solution  (assign  (54,  55)  to  02  and  (si,  sg)  to  ai)  for  both  oi  and  02  which  resolves 
conflicts.  So  in  step  2,  new  values  are  assigned.  Since  02  and  04  still  have  conflicts  on  54,  02  propagates 
the  negotiation.  But  note  that  04  has  only  two  resources  available,  the  only  partial  solution  for  02  and  04  is 
to  assign  (s2,  ■§4)  to  04  and  (si,  55)  to  02.  So  in  step  2,  new  values  are  assigned.  Now,  if  ai  is  to  propagate 
the  negotiation,  it  will  find  that  the  same  situation  in  step  2  as  in  step  0.  An  oscillation  is  detected  which 
makes  the  algorithm  exit  without  finding  the  possible  solution. 

Although  sequential  SPAM  is  not  complete,  our  experiments  show  that  the  possibility  for  the  algorithm 
to  be  complete  is  very  high,  especially  in  under-constrained  situations.  The  experiment  is  set  up  with  10 
agents  and  20  agents.  The  number  of  resources  are  30  and  60  respectively.  Each  task  has  a  fixed  objective 
level  of  3,  which  is  the  best  possible  in  both  situations.  The  availability  of  resources  to  agents  varies  from 
0.1  to  0.9.  The  availability  here  simply  means  the  probability  that  any  resource  can  be  used  by  an  agent. 
For  example,  in  our  experimental  setting  of  10  agents  and  30  resources,  when  availability  equals  0.5,  on 
average  each  agent  will  have  15  resources  to  use.  This  implies  that  when  the  availability  increases,  the 
constrainedness  of  the  problem  decreases.  Given  a  set  of  agents,  a  set  of  resources,  and  the  availability  p, 
each  problem  instance  in  our  experiment  is  generated  by  randomly  adding  an  edge  between  an  agent  and  a 
resource  with  the  probability  p.  An  edge  between  an  agent  and  a  resource  simply  means  that  the  resource 
is  available  to  the  agent. 

Figure  28  shows  the  ratio  of  the  problems  solved  by  sequential  and  synchronous  SPAM  algorithms  over 
10,000  problem  instances,  with  different  resource  availability.  Since  the  synchronous  SPAM  algorithm  is 
complete,  the  results  of  synchronous  SPAM  give  a  baseline  for  sequential  SPAM.  The  result  for  the  10 
agents  case  shows  that  when  the  resource  availability  is  bigger  than  0.4,  most  of  the  problems  are  solvable; 
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Figure  28:  Number  of  problems  solved  by  each  algorithm 

whereas  when  the  resource  availability  is  less  than  0.2,  most  of  the  problem  are  unsolvable.  As  with  other 
CSPs,  it  is  expected  that  the  hardest  instances  of  this  negotiation  problem  are  more  likely  to  occur  when 
the  resource  availability  is  between  0.2  and  0.4,  where  around  half  of  the  problems  are  solvable.  Notice 
that  when  the  problem  size  increases  to  20  agents,  the  transition  from  the  region  with  most  unsolvable 
problems  to  the  region  with  most  solvable  problems  is  even  sharper.  We  suspect  that  a  phase  transition 
may  exist  in  the  negotiation  problem. 

Figure  29  shows  the  ratio  of  completeness  of  the  sequential  SPAM  algorithm  as  the  resource  avail¬ 
ability  of  the  problem  increases  or  the  constrainedness  of  the  problem  decreases.  This  figure  comes  from 
the  same  results  plotted  in  Figure  28.  Each  data  point  equals  to  the  number  of  problems  solved  by  the  se¬ 
quential  SPAM  algorithm  divided  by  the  number  of  problems  solved  by  the  synchronous  SPAM  algorithm. 
Since  synchronous  SPAM  is  a  complete  algorithm,  the  ratio  directly  reflects  the  ratio  of  completeness  of 
sequential  SPAM.  The  result  shows  that  as  the  resource  availability  increases,  the  completeness  of  sequen¬ 
tial  SPAM  increases  as  well.  For  the  20  agents  problem,  when  the  availability  is  greater  than  0.4,  the 
completeness  ratio  of  sequential  SPAM  is  more  than  half.  When  the  resource  availability  increases  beyond 
0.6,  more  than  80%  of  the  solvable  problem  instances  are  solved  by  the  sequential  SPAM  algorithm.  This 
simply  indicates  that  the  original  SPAM  protocol  is  well  suited  for  under-constrained  problems,  having  a 
very  high  possibility  to  be  complete. 

3.4.2  Time  complexity 

The  sequential  SPAM  algorithm  sacrifices  completeness  for  computation  time,  as  to  be  verified  by  exper¬ 
iments.  Again,  we  run  both  sequential  and  synchronous  SPAM  algorithms  on  10,000  problem  instances 
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Figure  29:  The  rate  of  completeness  for  sequential  SPAM 

with  different  resource  availabilities.  The  problem  settings  are  still  10  agents  with  30  resources.  Figure  30 
plots  the  average  total  CPU  time  of  each  algorithm  for  these  10,000  instances  in  second.  Both  experiments 
were  on  a  linux  machine  with  756  MB  memory  and  an  AMD  1.4GHZ  processor. 

The  result  of  the  synchronous  SPAM  algorithm  shows  a  phenomenon  similar  to  that  of  phase  transi¬ 
tions.  It  takes  more  CPU  times  on  problems  with  resource  availability  at  0.3  and  0.4  than  the  problems  in 
the  other  resource  availability.  The  results  here  are  consistent  with  the  results  in  Figure  28  which  shows  that 
the  problems  around  0.3  and  0.4  availability  are  located  in  the  middle  of  a  phase  transition  on  solubility, 
and  thus  are  harder  to  determine  quickly  if  they  are  solvable  or  not. 

However,  the  result  of  the  sequential  SPAM  algorithm  does  not  show  any  phase-transition  phenomenon. 
Its  CPU  time  smoothly  increases  with  the  resource  availability.  Under  all  the  availability  settings  we  con¬ 
sidered,  the  sequential  SPAM  algorithm  takes  less  CPU  time  than  the  synchronous  SPAM  algorithm.  One 
explanation  is  that  the  sequential  algorithm  is  able  to  give  up  searching  for  a  solution  sooner  on  hard 
problems  than  the  synchronous  algorithm  so  that  the  former  finishes  faster  than  the  latter.  Another,  minor 
reason  is  that  the  sequential  SPAM  algorithm  has  a  lower  overhead  in  each  step. 

One  simple  implication  of  these  results  is  that  there  exists  a  tradeoff  between  the  completeness  and  time 
complexity  of  a  protocol.  Adding  additional  features  such  as  those  we  introduced  in  the  synchronous  algo¬ 
rithm  may  make  a  protocol  complete,  but  meanwhile  may  decrease  the  time  performance  of  the  protocol. 
For  many  soft-constrained  problems  in  practice  where  the  completeness  is  not  crucial,  using  incomplete 
algorithms  or  protocols  such  as  the  SPAM  protocol  seems  to  be  the  right  choice. 
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Figure  30:  The  CPU  time  of  each  algorithm 

3.4.3  Convergency  and  performance 

The  sequential  and  synchronous  algorithms  cannot  guarantee  to  always  turn  the  current  state  into  a  better 
one  after  each  step  of  negotiation.  For  asynchronous  SPAM,  during  the  stage  1,  when  an  agent  commits 
a  local  solution  that  has  no  conflict  with  partial  solutions,  this  new  solution  may  introduce  more  conflicts 
with  tentative  solutions  than  the  conflicts  it  can  reduce.  Consequently,  the  overall  conflicts  are  increased 
after  one  step  of  negotiation.  Similar  example  can  also  be  found  in  stage  2  of  the  sequential  SPAM 
algorithm.  However,  both  algorithms  are  able  to  terminate  after  a  finite  number  of  steps.  Therefore,  both 
algorithms  will  finally  converge  to  some  specific  values.  For  the  synchronous  SPAM  algorithm,  this  value 
is  0  if  the  problem  is  solvable  as  the  algorithm  is  complete,  while  for  the  sequential  SPAM  algorithm,  this 
value  may  not  necessarily  be  0. 

Here,  we  are  particularly  interested  in  the  rate  of  convergence  of  these  two  algorithms,  which  basically 
measures  how  fast  the  protocols  improve  solution  quality.  Figure  3 1  shows  the  experimental  results  with 
10  agents  and  30  resources  averaging  over  10,000  problem  instances,  indicating  that  the  convergency 
speed  of  the  sequential  algorithm  is  normally  better  than  that  of  the  synchronous  algorithm.  This  result  is 
not  surprising.  If  we  compare  the  two  algorithms,  we  will  find  that  in  each  step,  the  synchronous  SPAM 
algorithm  tries  to  resolve  the  conflicts  with  only  partial  solutions,  whereas  the  sequential  SPAM  tries  to 
resolve  the  conflicts  with  all  the  solutions.  This  means  that  in  each  step,  the  sequential  SPAM  is  likely  to 
resolve  more  conflicts  than  the  synchronous  SPAM,  so  that  its  convergency  speed  may  be  faster.  Figure  32 
also  shows  the  results  on  some  larger  problems  with  20  agents  and  30  agents. 

Based  on  these  results,  we  find  the  following  interesting  facts.  Despite  its  incompleteness,  when  the 
sequential  SPAM  algorithm  terminates,  the  final  solutions  are  almost  always  near  optimal.  Specifically,  the 


46 


10  Agents  30  Resources  Availability  =  0.2 


10  Agents  30  Resources  Availability  =  0.5 


SynSPAM  deviation 
SynSPAM  mean 
SeqSPAM  deviation 
SeqSPAM  mean 


10 

Number  of  steps 


100 


Figure  31:  The  convergency  speed  of  two  algorithms 


20  Agents  60  Resources  Availability  =  0.2  30  Agents  90  Resources  Availability  =  0.2 


Figure  32:  The  convergency  speed  of  two  algorithms 

average  solutions  from  sequential  SPAM  are  only  a  few  conflicts  more  than  that  of  synchronous  SPAM,  es¬ 
pecially  in  the  under-constrained  situations.  This  is  summarized  in  Table  2,  which  shows  the  performance 
results  of  two  algorithms  on  tracking  problems  with  different  problem  sizes  and  resource  availabilities. 
Each  data  point  in  the  table  shows  the  average  number  of  conflicts  unsolved  after  100  step  execution  of  the 
algorithms,  averaged  over  10,000  random  problem  instances.  Based  on  the  table,  most  of  the  results  of  the 
sequential  algorithm  are  comparative  to  the  synchronous  algorithm,  especially  in  the  under-constrained 
cases.  For  10-agent  problems,  the  biggest  performance  gap  occurs  when  the  resource  availability  is  equal 
to  0.3,  which  is  located  near  the  middle  of  a  phase  transition  on  solubility.  For  20  agent  problems,  the 
phase  transition  point  shifts  to  the  availability  of  0.2,  and  correspondingly  the  biggest  performance  gap 
shifts  as  well.  In  all  other  regions  apart  from  phase  transition,  the  performance  differences  between  the 
two  algorithms  are  trivial. 

In  light  of  all  the  above  results  and  the  results  on  the  completeness,  we  can  conclude  that  for  many 
real-time  applications,  it  is  reasonable  to  give  up  the  total  completeness  of  SPAM  protocol  in  favor  of  a 
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Table  2:  Mean  Conflicts  Over  10,000  problem  instances  After  100  Steps 


size 

algo. 

resource  availability 

0.2  0.3  0.4  0.5  0.6  0.8 

10  agents 

Syn.  SPAM 
Seq.  SPAM 

4.2956  0.8491  0.1761  0.0173  0.0016  0 

5.9764  3.0456  1.3994  0.6950  0.2799  0.0409 

20  agents 

Syn.  SPAM 
Seq.  SPAM 

0.7201  0.0377  0.0016  0  0  0 

5.9953  1.9827  0.8805  0.4009  0.1965  0.0342 

faster  convergency  to  a  good  enough  solution. 

3.4.4  Scalability 
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Figure  33:  Synchronous  SPAM  scalability 

Another  important  aspect  of  performance  analysis  of  a  protocol  is  the  study  of  how  the  performance 
varies  as  problem  size  increases,  i.e.,  how  the  protocol  scales.  This  feature  is  particularly  important  in 
applications  such  as  sensor  networks  in  ANTs  domains,  since  such  systems  can  easily  have  hundreds  or 
thousands  of  sensors.  A  good  scalability  means  that  system  performance  does  not  degrade  disproportion- 
ally  with  the  size  of  the  system  or  does  not  degenerate  at  all  when  the  number  of  components  increases. 
Here  we  are  interested  in  answering  the  following  question:  will  the  number  of  negotiation  steps  that 
SPAM  protocol  takes  to  find  a  solution  dramatically  increase  as  the  number  of  agents  increases? 

Figure  33  shows  the  experimental  results  for  the  synchronous  SPAM  protocol,  in  which  the  number  of 
agents  varies  from  5  to  25,  and  the  average  number  of  resources  available  to  each  agent  varies  from  6  to 
30.  The  total  number  of  resources  is  three  times  of  the  number  of  agents  in  each  of  1 ,000  random  problem 
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Figure  34:  Sequential  SPAM  scalability 

instances.  In  other  words,  the  objective  level  for  each  agent  is  set  to  3.  Note  that  for  the  5-agent  problems, 
the  maximum  number  of  resources  available  to  each  agent  is  15.  We  simply  extend  the  data  on  15  to  30 
in  the  5-agent  case  to  make  the  results  easy  to  show.  In  Figure  33,  each  data  point  represents  the  average 
number  of  steps  over  the  solvable  problems  of  1 ,000  runs.  From  the  results,  we  can  see  that  the  scalability 
of  synchronous  SPAM  is  super  linear.  Particularly  around  the  phase  transition  area,  the  performance  of  the 
protocol  substantially  degrades. 

Figure  34  shows  the  experimental  results  on  the  sequential  SPAM  protocol  with  the  same  experiment 
setup  as  above.  Again  each  data  point  represents  the  average  result  over  the  1,000  solvable  problem 
instances.  Comparing  to  the  results  in  Figure  33,  Figure  34  shows  that  the  scalability  of  sequential  SPAM 
is  better  than  the  scalability  of  synchronous  SPAM.  However,  based  on  these  results,  we  are  unable  to 
affirm  that  the  scalability  of  sequential  SPAM  is  linear.  Nevertheless,  the  results  indicate  that  the  sequential 
SPAM  algorithm  does  not  seems  to  have  a  substantial  degrading  region  around  the  phase-transition  area 
similar  to  the  synchronous  SPAM  algorithm.  We  should  also  note  that  at  each  point  in  Figure  34,  fewer 
problems  are  completely  solved  than  the  corresponding  point  in  Figure  33. 

3.4.5  Summary 

In  this  section,  we  experimentally  analyze  the  properties  and  performance  of  two  distributed  search  al¬ 
gorithms  derived  from  a  recently  developed  cooperative  negotiation  protocol  for  resource  allocation  in 
networks  of  distributed  sensors.  The  sequential  version  simulates  the  protocol  as  close  as  possible,  while 
the  synchronous  version  adds  some  additional  features  to  ensure  the  completeness.  The  experimental  re¬ 
sults  on  these  algorithms  help  us  understand  the  properties  of  the  original  protocol.  Specifically,  our  results 
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show  that  the  SPAM  protocol  is  not  complete  in  terms  of  finding  a  solution.  However  it  has  a  high  proba¬ 
bility  of  finding  a  solution  if  one  exists,  especially  in  under-constrained  situations.  The  SPAM  protocol  is 
able  to  give  up  sooner  on  the  hard  problems  so  as  to  finish  fasfer  fhan  fhe  complefe  profocols.  The  profocol 
converges  well  in  fhe  sense  fhaf  if  can  find  good  low-confiicf  solufions  wifhin  a  small  number  of  sfeps.  The 
solufion  qualify  is  comparafive  fo  fhe  complefe  profocol  in  mosf  of  fhe  cases.  The  profocol  seems  fo  scale 
very  well  as  fhe  number  of  agenfs  and  fhe  number  of  resources  increase. 

Wifh  all  fhese  resulfs  above  and  fhe  facl  fhaf  fhe  SPAM  profocol  is  normally  used  in  a  dynamic  envi- 
ronmenf,  if  is  reasonable  fo  give  up  fhe  complefeness  of  fhe  profocol  in  favor  of  compufafional  complexify, 
convergency  speed,  and  scalabilify.  Nofe  fhaf  in  a  dynamic  environmenf,  even  if  fhe  profocol  finally  finds 
fhe  optimal  solution  wifh  a  complefe  search,  fhe  problem  could  have  already  changed  and  fhe  optimal 
solufion  may  no  longer  be  relevanf.  Therefore,  making  fhe  righf  Iradeoff  befween  solution  qualify  and 
compufafional  lime  is  crilical,  and  fhe  SPAM  negolialion  profocol  seems  fo  make  such  a  Iradeoff  very  well 
for  real-lime  moving  largel  fracking. 

3.5  Conclusions  and  Discussions 

Motivated  fo  undersland  fhe  properlies  of  a  recenlly  developed  cooperalive  negolialion  profocol,  we  pro¬ 
posed  an  approach  fo  analyzing  dislribuled  negolialion  melhods.  In  Ibis  approach,  we  view  and  formulaic 
a  negolialion  profocol  as  a  dislribuled  search,  and  fhen  experimenlally  investigate  fhe  properlies  and  per¬ 
formance  of  fhe  search  algorilhms  fo  help  draw  conclusions  on  fhe  original  profocol.  We  demonslraled 
Ibis  approach  on  fhe  Scalable  Profocol  for  Anylime  Multi-level  (SPAM)  negolialion  profocol  for  allocaling 
resources  among  a  sel  of  cooperative  dislribuled  sensors.  In  addifion  fo  fhe  conlribulion  on  a  negolialion 
profocol  ilself,  we  subslanlially  extended  fhe  well  known  nolion  of  dislribuled  AI  as  dislribuled  search  [64]. 
We  nol  only  viewed  a  dislribuled  cooperalive  negolialion  as  a  dislribuled  search,  we  also  proposed  fo  use 
search  as  a  simulation  fool  fo  analyze  negolialion  protocols.  As  we  demonslraled  in  Ibis  section.  Ibis 
approach  can  overcome  many  difficulties  inherenl  to  a  negotiation  protocol  lhal  is  hard  to  investigate  an¬ 
alytically.  We  believe  lhal  Ibis  approach  is  general  and  can  be  carried  over  to  analyzing  olher  dislribuled 
problems  and  slralegies. 

In  reflecting  on  Ihe  experimenlal  resulls  on  Ihe  performance  of  Ihe  SPAM  negotiation  protocol  oblained 
in  Ibis  section,  il  seems  lhal  high-qualily  global  solutions  to  a  dislribuled  problem  can  be  achieved  wilhoul 
a  global  conlrol  bul  wilh  a  negotiation  protocol  in  which  agenls  rely  on  information  reslricled  to  Iheir 
small  neighborhoods.  Wilh  a  small  localily  of  information  sources,  anytime  performance  of  a  system  can 
also  be  significanlly  improved.  An  imporlanl  lesson  we  have  learnl  from  Ihe  resulls  of  Ibis  section  is 
lhal  propagating  information  among  neighboring  agenls  and  using  information  in  a  small  neighborhood 
vicinity  may  be  a  good  general  slralegy  for  dislribuled  problem  solving. 
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4  Phase  Transitions  and  Backbones  of  the  Asymmetric  Traveling  Salesman 


Phase  transitions  of  combinatorial  problems  and  threshold  behavior  similar  to  phase  transitions  in  combi¬ 
natorial  algorithms  have  drawn  much  attention  in  recent  years  [37,  44,  70].  Introduced  first  in  the  so-called 
spin  glass  theory  [75]  in  physics,  phase  transition  refers  to  such  a  phenomenon  of  a  system  in  which  some 
global  properties  change  rapidly  and  dramatically  when  a  control  parameter  goes  across  a  critical  value. 
A  daily-life  example  of  phase  transitions  is  water  changing  from  ice  (solid  phase)  to  water  (liquid  phase) 
to  steam  (gas  phase)  when  the  temperature  increases.  It  has  been  shown  that  many  combinatorial  decision 
problems  have  phase  transitions,  including  Boolean  satisfiability  [14,  77,  43,  88,  78],  graph  coloring  [14], 
and  the  symmetric  Traveling  Salesman  (deciding  the  existence  of  a  complete  tour  of  vising  a  given  set  of 
cities  with  a  cost  less  than  a  specified  value)  [33].  Phase  fransifion  can  be  used  fo  characferize  fypical-case 
properfies  of  difficull  combinatorial  problems  [37,  70].  The  hardesf  problem  insfances  of  mosf  decision 
problems  appear  very  often  af  fhe  poinfs  of  phase  fransifions.  Therefore,  phase  fransifions  have  been  used 
fo  help  generafe  fhe  hardesf  problem  insfances  for  fesfing  and  comparing  algorifhms  for  decision  prob¬ 
lems  [1,  14,  77]. 

Anofher  imporfanf  and  useful  concepf  for  characferizing  combinaforial  problems  is  fhaf  of  back¬ 
bone  [61,  78].  A  backbone  variable  refers  fo  such  a  variable  fhaf  has  a  fixed  value  among  all  optimal 
solufions  of  a  problem;  and  all  such  backbone  variables  are  collecfively  referred  fo  as  fhe  backbone  of  fhe 
problem.  If  a  problem  has  a  backbone  variable,  an  algorifhm  will  nol  find  a  solufion  fo  fhe  problem  unfil 
fhe  backbone  variable  is  sef  fo  ifs  correcf  value.  Therefore,  fhe  fracfion  of  backbone,  fhe  percenfage  of 
variables  being  in  fhe  backbone,  reflecfs  fhe  consfrainedness  of  fhe  problem  and  direcfly  affecls  an  algo¬ 
rifhm  searching  for  a  solufion.  The  larger  a  backbone,  fhe  more  fighfly  consfrained  fhe  problem  becomes. 
As  a  resulf  if  is  more  likely  for  an  algorifhm  fo  sef  a  backbone  variable  fo  a  wrong  value,  which  may 
consequenfly  require  a  large  amounf  of  compulation  to  recover  from  such  a  mislake. 

However,  fhe  research  on  fhe  phase  fransifions  and  (particularly)  backbones  of  optimization  problems 
is  limifed,  which  is  in  sharp  confrasl  wifh  fhe  numerous  sfudies  of  fhe  phase  fransifions  and  backbones 
of  decision  problems,  represented  by  Boolean  salisfiabilily  (e.g.,  [14,  77,  43,  88,  78]).  An  early  work  on 
fhe  symmefric  TSP  infroduced  fhe  concepf  of  backbones  [61].  However,  if  has  lefl  fhe  quesfion  whefher 
fhere  exisfs  a  phase  fransifion  of  fhe  TSP,  fhe  oplimizalion  version  of  fhe  problem  fo  be  specific,  open 
since  1985.  One  of  fhe  besf  (rigorous)  phase-fransifion  resulfs  was  obfained  on  number  partitioning  [10], 
an  optimization  problem.  However,  fhe  phase  fransifion  analyzed  in  [10],  also  experimenfally  in  [34,  35], 
is  fhe  existence  of  a  perfecl  parfifion  for  a  given  sef  of  integers,  which  is  in  essence  a  decision  problem. 
In  addifion,  [34,  35]  also  sfudied  fhe  phase  fransifions  of  fhe  size  of  oplimal  2-way  parfifion.  The  rela- 
lionship  belween  fhe  phase  fransifions  of  salisfiabilily,  a  decision  problem,  and  maximum  salisfiabilily,  an 
oplimizalion  problem,  was  sfudied  in  [108].  If  was  experimenfally  shown  fhaf  fhe  backbone  of  maximum 
Boolean  salisfiabilily  also  exhibils  phase  fransifions,  emerging  from  nonexislence  fo  almosl  a  full  scale 
abruplly  and  dramalically.  In  addition,  fhe  relalionship  belween  backbones  and  average-case  algorilhmic 
complexify  has  also  been  considered  [90]. 

In  Ibis  seclion,  we  investigate  fhe  phase  fransifions  of  fhe  asymmelric  Traveling  Salesman  Problem. 
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Our  results  answer  the  long-standing  open  question  of  [61]  based  on  the  more  general  form  of  the  problem. 
The  Traveling  Salesman  Problem  (TSP)  [38,  63]  is  an  arehetypieal  eombinatorial  optimization  problem 
and  one  of  the  first  NP-hard  problems  studied  [52].  Many  eoneepts,  sueh  as  baekbone  [61],  and  general 
algorithms,  sueh  as  braneh-and-bound  [66],  loeal  seareh  [65]  and  simulated  annealing  [86]  were  first 
introdueed  and  studied  using  the  TSP.  The  problem  is  also  very  often  a  touehstone  for  eombinatorial 
algorithms.  Furthermore,  the  faet  that  many  real-world  problems,  sueh  as  scheduling  and  routing,  can  be 
cast  as  TSPs  has  made  the  problem  of  practical  importance.  In  this  section,  we  consider  the  asymmetric 
TSP  (ATSP),  where  a  distance  from  one  city  to  another  may  not  be  necessarily  the  same  as  the  distance 
on  the  reverse  direction.  The  ATSP  is  more  general  and  most  ATSPs  are  more  difficult  to  solve  than  their 
symmetric  counterparts  [51]. 

Specifically,  using  uniformly  random  problem  instances  of  up  to  1,500  cities,  we  empirically  reveal  that 
the  average  optimal  tour  length,  the  accuracy  of  the  most  effective  lower-bound  function  for  the  problem 
(the  assignment  problem  [69]),  and  the  backbone  of  the  ATSP  undergo  sharp  phase  transitions.  The  control 
parameter  is  the  precision  of  intercity  distances  which  is  typically  represented  by  the  maximum  number 
of  digits  for  the  distances.  Note  that  these  results  are  algorithm  independent  and  are  properties  of  the 
problem.  Furthermore,  we  show  that  the  average  computational  cost  of  the  well-known  braneh-and-bound 
subtour  elimination  algorithm  [4,  7,  92]  for  the  ATSP  exhibits  a  phase-transition  or  threshold  behavior 
in  which  the  computational  cost  grows  abruptly  and  dramatically  from  easy  to  difficult  as  the  distance 
precision  increases.  Our  results  lead  to  a  practical  guidance  on  how  to  generate  large,  difficult  random 
problem  instances  for  the  purpose  of  algorithm  comparison. 

It  is  worthwhile  to  mention  that  besides  the  results  in  [61]  there  are  two  additional  pieces  of  early  em¬ 
pirical  work  related  to  the  phase  transitions  of  the  Traveling  Salesman.  The  research  in  [113]  investigated 
the  effects  of  two  different  distance  distributions  on  the  average  complexity  of  the  subtour  elimination 
algorithm  for  the  asymmetric  TSP.  The  main  result  is  that  the  average  complexity  of  the  algorithm  is  con¬ 
trolled  by  the  number  of  distinct  distances  of  a  random  asymmetric  TSP.  We  will  extend  this  result  further 
in  Section  4.5.  However,  we  need  to  caution  that  these  results  are  algorithm  specific,  which  may  not 
necessarily  reflect  intrinsic  features  of  the  underlying  problem.  The  research  in  [33]  studied  the  decision 
version  of  the  symmetric  TSP.  Specifically,  it  analyzed  the  probability  that  a  tour  whose  length  is  less  than 
a  specific  value  exists  for  a  given  random  symmetric  euclidean  TSP,  showing  that  the  probability  has  a 
one-to-zero  phase  transition  as  the  length  of  the  targeting  tour  increases.  Note  that  this  phase-transition 
result  does  not  address  the  open  question  of  [61]  which  is  on  the  optimization  version  of  the  problem.  The 
experimental  results  of  [33]  also  showed  that  the  computational  cost  of  a  braneh-and-bound  algorithm, 
which  unfortunately  was  not  specified  in  this  section,  exhibits  an  easy-hard-easy  pattern. 

This  section  is  organized  as  follows.  In  Section  4. 1,  we  describe  the  ATSP  and  a  related  problem  called 
assignment  problem.  We  then  investigate  the  parameter  that  controls  phase  transitions  in  Section  4.2,  and 
study  various  phase  transitions  of  the  ATSP  in  Section  4.3.  In  Section  4.5,  we  describe  the  well-known 
subtour  elimination  algorithm  for  the  ATSP,  and  analyze  the  threshold  behavior  of  this  algorithm.  We 
discuss  related  work  in  Section  4.6  and  finally  conclude  in  Section  4.7. 
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4.1  The  Asymmetric  Traveling  Salesman  and  Assignment  Problem 

Given  n  cities  and  the  distance  or  cost  between  each  pair  of  cities,  the  Traveling  Salesman  Problem  (TSP) 
is  to  find  a  minimum-cost  tour  that  visits  each  city  once  and  returns  to  the  starting  city  [38,  63].  When 
the  distance  from  city  i  to  city  j  is  the  same  as  the  distance  from  j  to  i,  the  problem  is  the  symmet¬ 
ric  TSP  (STSP).  If  the  distance  from  city  i  to  city  j  is  not  necessarily  equal  to  the  reverse  distance,  the 
problem  is  the  asymmetric  TSP  (ATSP).  The  ATSP  is  more  difficult  than  the  STSP,  with  respect  to  both 
optimization  and  approximation  [51].  The  TSPs  are  important  NP-hard  problems  [32,  52]  and  have  many 
practical  applications.  Many  difficult  combinatorial  optimization  problems,  such  as  vehicle  routing,  work¬ 
shop  scheduling  and  computer  wiring,  can  be  formulated  and  solved  as  the  TSPs  [38,  63]. 

The  ATSP  can  be  formulated  as  an  integer  linear  programming  problem.  Let  V  be  the  set  of  n  cities, 
A  the  set  of  all  pairs  of  cities,  and  D  =  {dij)  the  distance  or  cost  matrix  specifying  the  distance  of  each 
pair  of  cities.  The  following  integer  linear  programming  formulation  of  the  ATSP  is  well  known: 


ATSP{D)  =  min'y^dijXjj,  (1) 

subject  to 

=  yjGV]  (2) 

i£V 

^Xij  =  l,  \fieV-,  (3) 

j&v 

<|5|-1,  yscv,s^(D-,  (4) 

ieS  j£S 

Xij>0,  yijGV  (5) 


where  variables  Xij  take  values  zero  or  one,  and  Xij  =  1  if  and  only  if  arc  (f,  j)  is  in  the  optimal  tour,  for  i 
and  j  in  V.  Constraints  (2)  and  (3)  restrict  the  in-degree  and  out-degree  of  each  city  to  be  one,  respectively. 
Constraints  (4)  impose  the  subtour  elimination  constraints  so  that  only  complete  tours  are  allowed. 

The  ATSP  is  closely  related  to  the  assignment  problem  (AP)  [69],  which  is  to  assign  to  each  city  i 
another  city  j,  with  the  distance  from  i  to  j  as  the  cost  of  the  assignment,  such  that  the  total  cost  of 
all  assignments  is  minimized.  The  AP  is  a  relaxation  of  the  ATSP  in  which  the  assignments  need  not 
form  a  complete  tour.  In  other  words,  by  removing  the  subtour  elimination  constraints  (4)  from  the  above 
representation,  we  have  an  integer  linear  programming  formulation  of  the  AP.  Therefore,  the  AP  cost 
is  a  lower  bound  on  the  ATSP  tour  cost.  If  the  AP  solution  happens  to  be  a  complete  tour,  it  is  also  a 
solution  to  the  ATSP.  While  the  ATSP  is  NP-hard,  the  AP  can  be  solved  in  polynomial  time,  in  O(n^)  to 
be  precise  [69]. 

4.2  The  Control  Parameter 

Consider  two  cases  of  the  ATSP,  one  with  all  the  intercity  distances  being  the  same  and  the  other  with 
every  intercity  distance  being  distinct.  In  the  first  case,  every  complete  tour  going  through  all  n  cities  is 
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an  optimal  tour  or  a  solution  to  the  ATSP.  There  is  no  baekbone  variable  sinee  removing  one  edge  from  an 
optimal  solution  will  not  prevent  finding  another  optimal  solution.  The  ATSP  in  this  ease  is  easy;  finding 
an  optimal  solution  does  not  require  any  seareh  at  all.  In  addition,  the  eost  of  the  optimal  solution  is  also 
a  eonstant,  whieh  is  n  times  of  the  intereity  distanee.  In  the  seeond  ease  where  all  distanees  are  distinet, 
every  eomplete  tour  eovering  all  n  eities  has  a  high  probability  to  have  a  distinet  eost.  Therefore,  an 
are  in  the  optimal  solution  is  almost  surely  a  baekbone  variable  and  removing  it  may  destroy  the  optimal 
solution.  In  addition,  it  is  expeeted  to  be  diffieult  to  find  sueh  an  optimal  solution  among  a  large  number 
of  suboptimal  solutions  in  this  ease. 

Therefore,  there  are  signifieant  differenees  between  the  above  two  extreme  eases.  One  of  the  most 
important  differenees  is  the  number  of  distinet  distanees  in  the  distanee  matrix  D.  More  preeisely,  many 
important  eharaeteristies  of  the  random  ATSP,  ineluding  the  size  of  baekbone  and  eomplexity,  are  deter¬ 
mined  by  the  fraetion  of  distinet  distanees  among  all  distanees.  We  denote  the  fraetion  of  distinet  distanees 
in  distanee  matrix  D  as  p.  We  are  partieularly  interested  in  determining  how  p  affeets  the  eharaeteristies 
of  the  ATSP  when  it  gradually  inereases  from  zero,  when  all  distanees  are  the  same,  to  one,  when  all 
distanees  are  distinet. 

In  praetiee,  however,  we  do  not  direetly  eontrol  the  number  or  the  fraetion  of  distinet  distanees  in 
matrix  D.  Besides  the  aetual  struetures  of  the  “layouts”  of  the  eities,  the  preeision  of  the  distanees  affeets 
the  number  of  distinet  distanees.  The  preeision  of  a  number  is  usually  represented  by  the  maximal  number 
of  digits  allowed  for  the  number.  This  is  even  more  so  when  we  use  a  digital  eomputer  to  solve  the  ATSP, 
whieh  typieally  has  32  bits  (or  4  bytes)  for  integers  or  64  bits  (or  8  bytes)  for  double  preeision  real  numbers. 
As  a  result,  the  number  of  digits  for  distanees  is  naturally  a  good  ehoiee  for  the  eontrol  parameter. 

The  effeet  of  a  given  number  of  digits  on  the  fraetion  of  distinet  distanees  in  distanee  matrix  D  is 
relative  to  the  problem  size  n.  Consider  a  matrix  D  with  distanees  uniformly  ehosen  from  integers 
{0,1,2,--  -  ,R  —  1},  where  the  range  R  is  determined  by  the  number  of  digits  b.  For  a  fixed  number 
of  digits  b,  the  fraetion  of  distinet  distanees  of  a  larger  matrix  D  is  obviously  smaller  than  that  of  a  smaller 
D.  Therefore,  the  eontrol  parameter  for  the  fraetion  p  of  distinet  distanees  of  D  must  be  a  funetion  of  the 
number  of  digits  b  and  the  number  of  eities  n,  whieh  we  denote  as  p{n,  b). 

To  find  the  eontrol  parameter,  eonsider  the  number  of  distinet  distanees  in  D  for  a  given  integer  range 
R.  The  problem  of  finding  the  number  of  distinet  distanees  is  equivalent  to  the  following  bin-ball  problem. 
We  are  given  M  balls  and  R  bins,  and  asked  to  plaee  the  balls  into  the  bins.  Eaeh  ball  is  independently  put 
into  one  of  the  bins  with  an  equal  probability.  We  are  interested  in  the  fraetion  of  bins  that  are  not  empty 
after  all  the  plaeements.  Here,  for  asymmetrie  TSP  M  =  n?  —  n  balls  eorrespond  to  the  total  number  of 
nondiagonal  distanees  of  matrix  D,  and  R  bins  represent  the  possible  integers  to  ehoose  from.  Sinee  eaeh 
ball  (distanee)  is  thrown  independently  and  uniformly  into  one  of  the  R  bins  (integers),  the  probability 
that  one  bin  is  not  empty  after  throwing  M  balls  is  1  —  (1  —  1/R)^ .  The  expeeted  number  of  oeeupied 
bins  (distinet  distanees)  is  simply  i?  (l  —  (1  —  1/i?)^).  Thus,  the  expeeted  fraetion  of  distinet  distanees 
in  matrix  D  is 


E[p{n,b)] 


R{1-{1-  l/R)^) 
M 


(6) 
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effective  number  of  digits 


(b)  rescaled  fraction  of  distinct  numbers 


Figure  35:  (a)  Average  fraction  of  distinct  distances  in  matrix  D,  p{n,  b),  controled  by  the  effective  number 
of  digits,  /3  =  61og]”Q^(n),  for  n  =  100,  500, 1000  and  1500.  (b)  Average  /3(n,  b)  after  finite-size  scaling, 
with  scaling  factor  (/3  —  j3c)  log^Q(n),  where  j3c  =  2. 


Note  that  if  M  or  n  is  fixed,  E[p{n,  6)]  — >  1  as  i?  — >  oo,  since  in  fhis  case  fhe  expecfafion  of  fhe  number 
of  disfincf  disfances  approaches  M.  On  fhe  ofher  hand,  when  R  is  fixed,  E[p{n,  b)]  — >  0  when  M  ox  n 
goes  fo  infinify,  since  all  of  a  finile  number  of  R  bins  will  be  occupied  by  an  infinite  number  of  balls  in 
fhis  case. 

Following  convenfion  in  pracfice,  we  use  decimal  values  for  disfances.  Thus  R  =  10^,  where  b  is  fhe 
number  of  digifs  for  disfances.  If  furns  ouf  fhaf  if  we  plof  p(n,  b)  againsf  6/  log]^Q(n),  if  will  have  relatively 
fhe  same  scale  for  differenl  problem  sizes  n.  This  is  shown  in  Figure  35(a).  This  means  fhaf  fhe  scaling 
function  for  fhe  effective  number  of  digifs  is  /(n)  =  log)”Q^(n).  Function  b/  log]^o(^)  thus  fhe  effective 
number  of  digits  fhaf  confrols  fhe  fraction  of  disfincf  disfances  in  mafrix  D,  which  we  denote  as  /3(n,  b). 
This  also  means  fhaf  fo  have  fhe  same  effective  number  of  digifs  (5  on  fwo  differenf  problem  sizes,  say  ni 
and  n2  wifh  ni  <  n2,  fhe  range  R  should  be  differenl.  On  fhese  fwo  problems,  R  needs  fo  be  nf  and  n^, 
respectively,  giving  <  n^- 

We  need  fo  poinl  ouf  fhaf  fhe  infeger  range  R  can  also  be  represented  by  a  number  in  ofher  bases,  such 
as  binary.  Which  base  fo  use  will  nof  affecf  fhe  resulfs  quanlifalively,  buf  infroduces  a  consfanl  factor  fo  fhe 
resulls.  In  facl,  since  b  =  log]^Q(i?),  where  R  is  fhe  range  of  integers  to  be  chosen,  f3{n,  b)  =  b/  log^o(^)  = 
log„(i?),  which  is  independenl  of  fhe  base  of  fhe  values  for  inlercily  disfances. 

If  is  interesting  to  note  fhaf,  confrolled  by  fhe  effective  number  of  digifs  b/  fhe  fraction  of 

disfincf  entities  p  has  a  properly  similar  fo  a  phase  transition,  also  shown  in  Figure  35(a).  The  larger  the 
problem,  the  sharper  the  transition,  and  there  exists  a  crossover  point  among  the  transitions  of  problems 
with  different  sizes.  We  may  examine  this  phase-transition  phenomenon  more  closely  using  finite-size 
scaling.  Finite-size  scaling  [5,  101]  is  a  method  that  has  been  successfully  applied  to  phase  transitions  in 
similar  systems  of  different  sizes.  Based  on  finite-size  scaling,  around  a  critical  parameter  (temperature) 
Tc,  problems  of  different  sizes  tend  to  be  indistinguishable  except  for  a  change  of  scale  given  by  a  power 
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law  in  a  characteristic  length,  which  is  typically  in  the  form  of  (T  —  Tc)v}^'“ ,  where  n  is  the  problem  size 
and  V  the  exponent  of  the  rescaling  factor.  Therefore,  finite-size  scaling  characterizes  a  phase  transition 
precisely  around  the  critical  point  Tc  of  the  control  parameter  as  the  problem  scales  to  infinity.  However, 
our  analysis  revealed  that  the  scaling  factor  has  a  large  exponent  of  9  [110],  indicating  that  the  phase 
transitions  in  Figure  35(a)  does  not  exactly  follow  the  power  law  finite-size  scaling. 

To  find  fhe  correcf  rescaled  confrol  parameter,  we  reconsider  (6).  As  n  ^  oo  and  disfance  range  R 
grows  wifh  problem  size  n,  i.e.,  i?  ^  oo  as  n  ^  oo,  we  can  rewrite  (6)  as 


lim  E[p{n,h)]  =  Yim  ^  (l  -  {{I  -  I / R)^)^ ^ 

oo.it— >00  it— >oo  iVl  \  / 


M  ' 


(V) 


where  fhe  second  equation  follows  lim^^oo(l  —  l/R)^  =  e~^.  Since  our  underlying  confrol  parameter 
is  fhe  number  of  digifs,  b  =  log^g(i?),  we  lake  x  =  logiQ{R/M).  Asymplolically  as  n  ^  oo,  M  ~  n^, 
which  leads  lo  x  =  log;^o(-^)  “  ^  logio(n)  =  {(5  —  2)  log^gC^)-  Using  x,  we  rewrite  (7)  as 


lim  E[p{n,  6)]  =  10^  fl  -  .  (8) 

n— >oo,/?— >oo  \  / 

The  rescaled  confrol  paramefer  as  n  ^  oo  for  fhe  expecled  number  of  disfincf  dislances  in  79  is  (/3  — 
2)  logiQ(n).  Therefore,  fhe  critical  poinf  is  2  and  fhe  rescaling  factor  is  logig(n).  The  rescaled  phase 
Iransifion  is  shown  in  Figure  35(b),  which  plols  p(n,  (/7  —  2)  logig(n)). 

Note  fhaf  fhe  number  of  digifs  used  for  inlercily  dislances  is  nolhing  bul  a  measuremenl  of  fhe  precision 
of  fhe  dislances.  The  larger  fhe  number  of  digifs,  fhe  higher  fhe  precision  becomes.  This  agrees  wifh  fhe 
common  practice  of  using  more  effective  digifs  to  gain  precision.  Therefore,  fhe  confrol  parameter  is  in 
lurn  delermined  by  fhe  precision  of  inlercily  dislances. 

Finally,  if  is  imporlanl  to  nole  lhal  even  Ihough  fhe  discussion  in  Ihis  seclion  focused  on  asymmelric 
cosl  mafrices  and  fhe  ATSP,  fhe  argumenls  apply  lo  symmelric  disfance  malrices  and  fhe  symmelric  TSP 
as  well.  Thai  is,  wifh  M  revised  to  (n^  —  n)/2,  asymptotically  as  R  and  n  goes  to  infinily,  logio(M)  ~ 
2  log^Q(n),  so  lhal  (/3  —  2)  logig(n)  is  also  a  rescaled  confrol  parameter  for  fhe  number  of  disfincf  dislances 
in  symmelric  cosl  malrices. 


4.3  Phase  Transitions 

Wifh  fhe  confrol  paramefer,  fhe  effeclive  number  of  digifs  /7(n,  b)  for  inlercily  dislances,  identified,  we  are 
now  in  a  posilion  to  investigate  possible  phase  Iransilions  in  fhe  ATSP  and  fhe  relaled  assignmenl  problem. 

To  sel  forlh  fo  invesligale  fhese  phase  Iransilions,  we  generaled  and  sludied  uniformly  random  problem 
inslances  wifh  100-,  200-,  300-  uplo  1,000-cities  and  1,500-cities.  Allhough  we  have  resulls  from  100-, 
200-,  300-,  up  to  1,000-cily  as  well  as  1,500-cily  problems,  to  make  fhe  resull  figures  readable,  we  only  use 
fhe  dala  from  100-,  500-,  1,000-  and  1,500-cily  problems  fo  reporl  fhe  resulls.  For  fhe  problem  inslances 
considered,  inlercily  dislances  are  independenlly  and  uniformly  chosen  from  {0,l,2,---,7?  —  Ijfora 
given  range  R,  which  is  conlrolled  by  Ihe  number  of  digils  b.  We  varied  b  from  1.0  to  6.0  for  inslances 
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(a)  ATSP  tour  cost 


(b)  rescaled  ATSP  tour  cost 


Figure  36:  (a)  Average  optimal  ATSP  tour  cost,  (b)  Scaled  and  normalized  average  optimal  tour  cost,  with 
rescaling  factor  {(3  —  (3c)  logio(?^)  /3c  =  1- 


with  up  to  1,000-cities  and  from  1.0  to  6.5  for  instances  with  1,500-cities.  The  digits  are  incremented  by 
0.1,  i.e.,  we  used  b  =  1.0, 1.1, 1.2,  •  •  • . 

4.3.1  Phase  transitions  in  the  ATSP 

We  are  particularly  interested  in  possible  phase  transitions  in  the  ATSP  cost,  phase  transitions  of  backbones 
and  phase  transitions  of  the  numbers  of  ATSP  tours.  The  results  on  backbone  can  shed  some  light  on  the 
intrinsic  tightness  of  the  constraints  among  the  cities  as  the  precision  of  distance  measurement  changes. 
The  ATSP  tour  cost 

There  is  a  phase  transition  in  the  ATSP  tour  cost,  ATSP{D),  under  the  control  parameter  (3,  the 
effective  number  of  digits  for  intercity  distances.  Figure  36(a)  shows  the  results  on  100-,  500-,  1,000-  and 
1,500-city  ATSP  instances,  averaged  over  10,000  instances  for  each  data  point.  The  reported  tour  costs 
are  obtained  by  dividing  the  integer  ATSP  tour  costs  by  n  x  (i?  —  1),  where  n  is  the  number  of  cities  and 
R  the  range  of  intercity  costs.  Equivalently,  an  intercity  distance  was  virtually  converted  to  a  real  value  in 
[0,1]. 

As  shown,  the  ATSP  tour  cost  increases  abruptly  and  dramatically  as  the  effective  number  of  digits 
increases,  exhibiting  phase  transitions.  The  transitions  become  sharper  as  the  problem  becomes  larger,  and 
there  exist  crossover  points  among  curves  from  different  problem  sizes.  By  finite-size  scaling,  we  further 
determine  the  critical  value  of  the  control  parameter  at  which  the  phase  transitions  occur.  Following  the 
discussion  in  Section  4.2,  the  scaling  factor  has  a  form  of  {(3  —  (3c)  logig(n).  Our  numerical  result  indicated 
that  (3c  =  1.02  ±  0.007.  We  thus  use  /3c  =  1  to  show  the  result  in  Figure  36(b).  It  is  worthwhile  to 
mention  that  the  AP  cost  follows  almost  the  same  phase-transition  pattern  of  the  control  parameter  shown 
in  Figure  35  with  a  slightly  different  critical  point  (data  not  shown  here). 

Backbone  and  number  of  optimal  solutions 
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(a)  fraction  of  backbone 


(b)  rescaled  fraction  of  backbone 


Figure  37:  (a)  Average  fraction  of  backbone,  (b)  Rescaled  average  backbone  fraction,  with  rescaling  factor 
(/3  -  Pc)  logio(?^)  and  Pc  =  1. 


We  now  turn  to  the  backbone  of  the  ATSP,  which  is  the  percentage  of  directed  arcs  that  appear  in 
all  optimal  solutions.  The  backbone  also  exhibits  phase  transitions  as  the  effective  number  of  digits  for 
distances  increases.  The  result  is  included  in  Figure  37(a),  where  each  data  point  is  averaged  over  10,000 
problem  instances.  The  rescaled  result  is  shown  in  Figure  37(b),  where  the  critical  point  Pc  =  1.  Inter¬ 
estingly,  the  phase-transition  pattern  of  the  backbone  is  almost  identical  to  that  of  the  fraction  of  distinct 
entities  in  the  distance  matrix,  shown  in  Figure  35,  and  that  of  the  ATSP  tour  cost,  shown  in  Figure  36. 

The  fraction  of  backbone  is  related  to  the  number  of  optimal  solutions  of  a  problem.  We  thus  examined 
the  total  number  of  optimal  solutions  of  the  ATSP.  This  was  done  on  small  ATSPs,  from  10  cities  to  150 
cities,  as  finding  all  optimal  solutions  on  larger  problems  is  computationally  too  expensive.  The  results 
are  averaged  over  100  trials  for  each  data  point.  As  shown  in  Figure  38,  where  the  vertical  axes  are  in 
logarithmic  scale,  the  number  of  optimal  tours  also  undergoes  a  phase  transition,  from  exponential  to  a 
constant,  as  the  number  of  digits  increases.  Note  that  when  the  number  of  digits  is  small,  it  is  very  costly 
to  find  all  optimal  solutions,  even  on  these  relatively  small  problems. 

The  fraction  of  backbone  captures  in  essence  the  tightness  of  the  constraints  among  the  cities.  As 
more  intercity  distances  become  distinct,  the  number  of  tours  of  distinct  lengths  increases.  Consequently, 
the  number  of  optimal  solutions  decreases  and  the  fraction  of  backbone  grows  inversely.  When  more  arcs 
are  part  of  the  backbone,  optimal  solutions  become  more  restricted.  As  a  result,  the  number  of  optimal 
solutions  decreases.  As  the  fraction  of  backbone  increases  and  approaches  one,  the  number  of  optimal 
solutions  decreases  and  becomes  one  as  well,  which  typically  makes  the  problem  of  finding  an  optimal 
solution  more  difficult. 

4.3.2  Existence  of  Hamiltonian  circuit  with  zero-cost  edges 

When  the  precision  of  intercity  distances  is  low,  it  is  likely  that  the  ATSP  tour  has  a  cost  of  zero, 
meaning  that  there  exists  an  Hamiltonian  circuit  consisting  of  zero-cost  arcs.  It  is  a  decision  problem  to 
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effective  number  of  bits 


rescaied  effective  number  of  digits 


Figure  38:  (a)  Average  number  of  optimal  ATSP  tours,  (b)  Rescaled  average  number  of  optimal  ATSP 
tours,  with  rescaling  factor  {(3  —  f3c)  logig(n)  and  /3c  =  1-39  ±  0.008. 


determine  if  an  Hamiltonian  circuit  exists  in  a  given  ATSP.  We  examined  this  decision  problem  using  the 
same  set  of  10,000  problem  instances  used  in  Figures  36  and  37.  The  result  is  shown  in  Figure  39.  Notice 
that  although  it  follows  the  same  rescaling  formula  of  (/3  —  /3c)  logio(?^)>  ^^e  critical  point  of  the  transition, 
/3o  =  0.865,  is  different  from  the  critical  point  of  /3c  =  1  for  the  phase  transitions  of  backbones  and  ATSP 
tour  cost,  as  shown  in  Figures  36  and  37. 

4.3.3  Quality  of  the  AP  lower-bound  function 

The  existence  of  Hamiltonian  circuits  of  zero-cost  arcs  also  indicates  that  when  the  number  of  digits  for 
intercity  distances  is  very  small,  for  example,  less  than  1.9  (or  R  «  80)  for  n  =  1, 500,  both  the  AP  and 
ATSP  costs  are  zero,  meaning  that  these  two  costs  are  the  same  as  well.  It  is  useful  to  know  how  likely  the 
AP  cost  is  equal  to  the  ATSP  tour  cost;  the  answers  to  this  issue  constitutes  the  first  step  to  the  elucidation 
of  the  accuracy  of  the  AP  lower-bound  cost  function. 

Given  a  random  distance  matrix  D,  how  likely  is  it  that  an  AP  cost  will  be  the  same  as  the  ATSP 
tour  cost  as  the  effective  number  of  digits  /3  increases?  We  answer  this  question  by  examining  the  prob¬ 
ability  that  an  AP  cost  AP{D)  is  equal  to  the  corresponding  ATSP  cost  ATSP{D)  as  /3  increases.  Fig¬ 
ure  40(a)  shows  the  results  on  100-,  500-,  1,000-  and  1,500-city  ATSP  instances,  averaged  over  the  same 
set  of  10,000  instances  for  Figure  36  for  each  data  point.  As  shown  in  the  figure,  the  probability  that 
AP{D)  =  ATSP{D)  also  experiences  abrupt  and  dramatic  phase  transitions.  Figure  40(b)  shows  the 
phase  transitions  after  finite-size  scaling,  with  critical  point  /3c  =  1.17  ±  0.005. 

The  results  in  Figure  40  also  imply  that  the  quality  of  the  AP  lower-bound  function  degrades  as  the 
distance  precision  increases.  The  degradation  should  also  follow  a  phase-transition  process.  This  is  verified 
by  Figure  41,  using  the  data  from  the  same  set  of  problem  instances.  Note  that  the  critical  point  of  the 
phase  transition  for  the  accuracy  of  AP  is  /3c  =  0.97,  which  is  different  from  the  critical  point  /3c  =  1.17 
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(a)  probability  of  Hamiltonian  circuit 


(b)  rescaied  prob.  of  Hamiltonian  circuit 


rescaled  effective  number  of  diaits 


Figure  39:  (a)  Probability  of  the  existence  of  Hamiltonian  circuits  with  zero  cost  arcs,  (b)  Rescaled 
probability  of  zero-cost  Hamiltonian  circuits,  with  rescaling  factor  {(5  —  Pc)  logio(?^)  Pc  =  0.865. 


for  the  phase  transition  of  the  probability  that  AP{D)  =  ATSP{D). 

4.3.4  How  many  phase  transitions? 

So  far,  we  have  seen  many  phase  transitions  on  different  features  of  the  ATSP  and  its  related  assignment 
problem.  Qualitatively,  all  these  phase  transitions  follow  the  same  transition  pattern,  meaning  that  they 
can  all  be  captured  by  the  same  finite-size  rescaling  formula  of  (/?  —  Pc)  logioC^T'),  where  Pc  is  a  critical 
point  depending  on  the  particular  feature  of  interest. 

It  is  interesting  to  note  that  the  critical  points  for  the  phase  transitions  of  the  ATSP  tour  costs  and 
backbone  fractions  are  all  at  /3c  =  1-  A  close  examination  also  indicates  that  these  two  phase  transitions 
follow  almost  the  same  phase  transition,  as  shown  in  Figure  42,  where  the  rescaled  curves  for  the  ATSP  tour 
cost  and  the  backbone  fraction  are  drawn  from  1,500-city  ATSP,  averaged  over  10,000  problem  instances. 

Except  the  close  similarity  of  the  phase  transitions  of  the  ATSP  tour  cost  and  the  fraction  of  backbones, 
the  other  phase  transitions  all  have  different  critical  points,  indicating  that  they  undergo  the  same  type  of 
phase  transitions  at  different  ranges. 

4.4  Asymptotic  ATSP  tour  length  and  AP  precision 

As  a  by-product  of  the  phase-transition  results,  we  now  provide  numerical  values  of  the  ATSP  cost,  the  AP 
cost  and  its  accuracy,  asymptotically  when  the  number  of  cities  grows.  We  attempt  to  extend  the  previous 
theoretical  results  on  the  AP  cost,  which  was  shown  to  asymptotically  approach  vr^/G  [3,  74],  and  the 
observations  that  the  relative  error  of  the  AP  lower  bounds  descreases  as  the  problem  size  increases  [4,  92]. 

Not  every  real  number  can  be  represented  in  a  digital  computer.  Thus,  it  is  infeasible  to  directly 
examine  a  theoretical  result  on  reals  using  a  digital  computer.  For  our  purpose,  on  the  other  hand,  the 
phase-transition  results  indicate  that  when  the  precision  of  the  intercity  distances  is  high  enough,  all  the 
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(a)  probability  AP(D)  =  ATSP(D) 


(b)  rescaled  probability  AP(D)=ATSP(D) 


Figure  40:  (a)  Average  probability  that  AP{D)  =  ATSP{D).  (b)  Average  probability  after  finite-size 
scaling,  with  rescaling  factor  (/3  —  f5c)  logio(^)  Pc  =  1-17  ±  0.005. 


quantities  of  the  ATSP  we  have  examined,  including  the  ATSP  cost,  the  AP  cost  and  its  precision  as  a 
lower-bound  cost  function,  as  well  as  the  backbone,  are  relatively  stable,  in  the  sense  that  they  do  not 
change  significantly  even  when  the  precision  of  intercity  distances  increases  further.  Therefore,  it  is  suffi¬ 
cient  to  use  a  high  distance  precision  to  experimentally  analyze  the  asymptotic  properties  of  the  ATSP  cost 
and  other  related  quantities. 

We  need  to  be  cautious  in  selecting  the  number  of  digits  for  intercity  distances.  As  we  discussed  in 
Section  4.2,  the  same  number  of  digits  for  intercity  distances  gives  rise  to  different  effective  numbers  of 
digits  on  problems  of  different  sizes.  Furthermore,  the  phase  transition  results  in  Section  2.3.2  indicate 
that  the  effective  numbers  of  digits  must  be  scaled  properly  in  order  to  have  the  same  effect  on  problems 
of  different  sizes  when  we  investigate  an  asymptotic  feature. 

Therefore,  in  our  experiments,  we  fixed  the  scaled  effective  number  of  digits  for  intercity  distances, 
(/3  —  /3c)  logiQ(n),  to  a  constant.  Based  on  our  phase-transition  results,  especially  that  on  the  control 
parameter  in  Figure  35,  we  chose  to  take  (/3  —  2)logig(n)  a  constant  of  2.1,  for  two  reasons.  First, 
(/3  —  2)  logig(n)  =  2.1  is  sufficiently  large  so  that  almost  all  distances  are  distinct,  regardless  of  problem 
size,  and  the  quantities  we  will  examine  will  not  change  substantially  after  the  finite-size  scaling.  Secondly, 
(/3  —  2)  log]^Q(n)  =  2.1  is  relatively  small  so  that  we  can  experiment  on  problems  of  large  sizes.  To  save 
memory  as  much  as  possible,  the  intercity  distances  are  integers  of  4  bytes  in  our  implementation  of  the 
subtour  elimination  algorithm.  Thus  the  number  of  digits  must  be  less  than  9.4  without  causing  an  overflow 
in  the  worst  case.  Using  (/3  —  2)  log^oC^)  =  2.1,  we  can  go  up  to  roughly  3,000-city  ATSPs. 

Table  3  shows  the  experimental  results,  with  up  to  3,000  cities,  on  the  average  AP  cost,  the  ATSP  tour 
cost,  and  accuracy  of  the  AP  cost  function  in  the  error  of  AP  cost  relative  to  the  ATSP  cost.  The  results 
are  averaged  over  10,000  instances  for  each  problem  size.  Based  on  the  results,  the  AP  cost  approaches 
to  1.6442  and  the  ATSP  cost  to  1.6446.  Note  that  the  experimental  AP  cost  of  1.6442  is  very  close  to  the 
theoretical  asymptotic  AP  cost  of  vr^/G  «  1.6449  [3,  74].  In  addition,  the  accuracy  of  AP  function  indeed 
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(b)  rescaled  accuracy  of  AP  function 


Figure  41:  (a)  Average  accuracy  of  AP  lower-bound  function,  measured  by  the  error  of  AP  cost  relative 
to  ATSP  cost,  (b)  normalized  and  rescaled  average  accuracy,  with  rescaling  factor  (/3  —  (5c)  logj^gC^) 

Pc  =  0.97. 


improves  as  the  problem  size  increases,  reduced  to  about  0.02548%  for  3,000-city  problem  instances.  This 
result  supports  the  previous  observations  [4,  92]. 

4.5  Threshold  Behavior  of  Suhtour  Elimination 

All  the  phase-transition  results  discussed  in  the  previous  section  indicate  that  the  ATSP  becomes  more 
constrained  and  difficult  as  the  distance  precision  becomes  higher.  In  this  section,  we  study  how  a  well- 
known  algorithm  for  the  ATSP,  branch-and-bound  subtour  elimination  [4,  7, 92],  behaves.  We  separate  this 
issue  from  the  phase  transition  phenomena  studied  before  because  what  we  will  consider  in  this  section  is 
the  behavior  of  a  particular  algorithm,  which  may  not  be  necessarily  a  feature  of  the  underlying  problem. 
Nevertheless,  this  is  still  an  issue  of  its  own  interest  because  this  algorithm  is  the  oldest  and  is  still  among 
the  best  known  methods  for  the  ATSP,  and  we  hope  that  a  better  understanding  of  an  efficient  algorithm 
for  the  ATSP  can  shed  light  on  the  typical  case  computational  complexity  of  the  problem. 

4.5.1  Branch-and-bound  subtour  elimination 

The  branch-and-bound  (BnB)  subtour  algorithm  elimination  [4,  7,  92]  solves  an  ATSP  in  a  state-space 
search  [82,  107]  and  uses  the  assignment  problem  (AP)  as  a  lower-bound  cost  function.  The  BnB  search 
takes  the  original  ATSP  as  the  root  of  the  state  space  and  repeats  the  following  two  steps.  First,  it  solves  the 
AP  of  the  current  problem.  If  the  AP  solution  is  not  a  complete  tour,  it  decomposes  it  into  subproblems  by 
subtour  elimination  that  breaks  a  subtour  by  excluding  some  arcs  from  a  selected  subtour.  As  a  subproblem 
is  more  constrained  than  its  parent  problem,  the  AP  cost  to  the  subproblem  must  be  as  much  as  that  of  the 
parent.  This  means  that  the  AP  cost  function  is  monotonically  nondecreasing.  While  solving  the  AP 
requires  0{nP)  computation  in  general,  the  AP  to  a  child  node  can  be  incrementally  solved  in  0{rP)  time 
based  on  the  solution  to  the  AP  of  its  parent. 
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Figure  42:  Simultanous  examination  of  the  phase  transitions  of  backbone  and  ATSP  tour  cost  on  1,500-city 
problems,  all  rescaled  with  (/3  —  1)  log^oC^t')- 


There  are  many  heuristics  for  selecting  a  subtour  to  eliminate  [4],  and  we  use  the  Carpaneto-Toth 
scheme  [13],  or  the  CT  scheme  for  short,  in  our  algorithm.  One  important  feature  of  the  CT  scheme  is 
that  it  generates  no  duplicate  subproblem  so  that  the  overall  search  space  is  a  tree.  One  example  of  this 
scheme  is  shown  in  Figure  43.  The  AP  solution  to  the  original  ATSP  contains  two  subtours  that  are  in  the 
root  of  the  tree  of  the  figure.  The  subtour  2  —  3  —  2  is  chosen  to  be  eliminated,  since  it  is  shorter  than 
the  other  subtour.  We  have  two  ways  to  break  the  selected  subtour,  i.e.,  excluding  directed  arc  (2, 3)  or 
(3, 2).  Assume  that  we  first  exclude  (2, 3)  and  then  (3,  2),  generating  two  subproblems,  nodes  A  and  B 
in  Figure  43.  When  generating  the  second  subproblem  B,  we  deliberately  include  (2,  3)  in  its  solution. 
By  including  the  arc  that  was  excluded  in  the  previous  subproblem  A,  we  force  to  exclude  in  the  current 
subproblem  B  all  the  solutions  to  the  original  problem  that  will  appear  in  A,  and  therefore  form  a  partition 
of  the  solution  space  using  A  and  B.  In  general,  let  £  be  the  excluded  arc  set,  and  Z  be  the  included  arc  set 
of  the  problem  to  be  decomposed.  Assume  that  there  are  t  arcs  of  the  selected  subtour,  {ei,  62,  •  •  •  ,  et}, 
that  are  not  in  I.  The  CT  scheme  decomposes  the  problem  into  t  child  subproblems,  with  the  fc-th  one 
having  excluded  arc  set  £k  and  included  arc  set  Z^,  such  that 


£k 

Ik 


=  £VJ  {efc}, 


|fc  =  l,2,---  ,f. 


(9) 


Since  is  an  excluded  arc  of  the  fe-th  subproblem,  G  £k,  and  it  is  an  included  arc  of  the  k  +  1-st 
subproblem,  G  Zk+i,  a  complete  tour  obtained  from  the  fe-th  subproblem  does  not  contain  arc  Sk,  while 
a  tour  obtained  from  the  A;  -|-  1-st  subproblem  must  have  arc  e^.  Thus  a  tour  from  the  k-th  subproblem 
cannot  be  generated  from  the  k  1-st  subproblem,  and  vice  versa.  In  summary,  the  state  space  of  the 
ATSP  under  BnB  using  the  CT  subtour  elimination  scheme  can  be  represented  by  a  tree  without  duplicate 
nodes. 

In  the  next  step,  the  algorithm  selects  as  the  current  problem  a  new  subproblem  from  the  set  of  active 
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n 

digifs 

AP  cosf 

ATSP  cosf 

relafive  AP  error  (%) 

200 

6.7021 

1.63533  ±0.00254 

1.64302  ±  0.00254 

0.46817  ±0.00970 

400 

7.3041 

1.63942  ±0.00180 

1.64311  ±0.00180 

0.22485  ±  0.00468 

600 

7.6563 

1.64072  ±0.00146 

1.64314  ±0.00145 

0.14765  ±0.00317 

800 

7.9062 

1.64227  ±0.00125 

1.64407  ±  0.00125 

0.10904  ±0.00237 

1,000 

8.1000 

1.64297  ±0.00114 

1.64441  ±0.00114 

0.08754  ±  0.00191 

1,200 

8.2584 

1.64284  ±0.00104 

1.64402  ±  0.00105 

0.07187  ±0.00158 

1,400 

8.3923 

1.64313  ±  0.00096 

1.64413  ±  0.00096 

0.06148  ±  0.00139 

1,600 

8.5082 

1.64319  ±  0.00090 

1.64405  ±  0.00090 

0.05276  ±0.00117 

2,000 

8.7021 

1.64382  ±0.00082 

1.64451  ±0.00082 

0.04231  ±  0.00095 

2,200 

8.7848 

1.64372  ±  0.00077 

1.64434  ±  0.00077 

0.03813  ±0.00085 

2,400 

8.8604 

1.64360  ±  0.00074 

1.64417  ±  0.00073 

0.03477  ±  0.00079 

2,600 

8.9299 

1.64429  ±  0.00071 

1.64481  ±0.00071 

0.03234  ±  0.00074 

2,800 

8.9943 

1.64382  ±0.00068 

1.64430  ±  0.00068 

0.02966  ±  0.00068 

3,000 

9.0542 

1.64421  ±  0.00065 

1.64463  ±  0.00065 

0.02548  ±  0.00061 

Table  3:  Numerical  results  on  AP  cost,  the  ATSP  cost  and  AP  error  relative  to  the  ATSP  cost,  in  percent. 
The  cost  matrices  are  uniformly  random.  Each  data  point  is  averaged  over  10,000  problem  instances.  In 
the  table,  n  is  the  number  of  cities,  digits  is  the  number  of  digits  for  intercity  distances,  and  all  numerical 
error  bounds  represent  95  percent  confidence  intervals. 


subproblems,  which  are  the  ones  that  have  been  generated  but  not  yet  expanded.  This  process  continues 
until  there  is  no  unexpanded  problem,  or  all  unexpanded  problems  have  costs  greater  than  or  equal  to  the 
cost  of  the  best  complete  tour  found  so  far. 

Thanks  to  its  linear-space  requirement,  we  use  depth-first  branch-and-bound  (DFBnB)  in  our  algo¬ 
rithm.  DFBnB  explores  active  subproblems  in  a  depth-first  order.  It  uses  an  upper  bound  a  on  the  optimal 
cost,  whose  initial  value  can  be  infinity  or  the  cost  of  an  approximate  solution,  such  as  one  obtained  by 
Karp’s  patching  algorithm  [53,  56],  which  repeatedly  patches  two  smallest  subtours  into  a  big  one  until 
a  complete  tour  forms.  Starting  at  the  root  node,  DFBnB  selects  a  recently  generated  node  x  to  examine 
next.  If  the  AP  solution  of  x  is  a  complete  tour,  then  x  is  a  leaf  node  of  the  search  tree.  If  the  cost  of  a 
leaf  node  is  less  than  the  current  upper  bound  a,  a  is  revised  to  the  cost  of  x.  If  x’s  AP  solution  is  not  a 
complete  tour  and  its  cost  is  greater  than  or  equal  to  a,  x  is  pruned,  because  node  costs  are  monotonic  so 
that  no  descendant  of  x  will  have  a  cost  smaller  than  x’s  cost.  Otherwise,  x  is  expanded,  generating  all  its 
child  nodes.  To  find  an  opfimal  goal  node  quickly,  fhe  children  of  x  should  be  searched  in  an  increasing 
order  of  fheir  cosfs.  In  ofher  words  we  use  node  ordering  fo  reduce  fhe  number  of  nodes  explored.  To  speed 
up  fhe  process  of  reaching  a  beffer,  possibly  opfimal,  solution,  we  also  apply  Karp’s  pafching  algorifhm  fo 
fhe  besf  child  node  of  fhe  currenf  node. 

Our  algorifhm  is  in  principle  fhe  same  as  fhaf  of  [12],  which  is  probably  fhe  besf  known  complefe 
algorifhm  for  fhe  ATSP.  The  main  difference  befween  fhe  fwo  is  fhaf,  due  fo  a  considerafion  on  space 
requiremenf,  we  use  depfh-firsf  search  while  [12]  used  besf-firsf  search. 
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Figure  43:  DFBnB  subtour  elimination  on  the  ATSR 

(a)  normalized  number  of  AP  calls  (b)  rescaled  normalized  #  of  AP  calls 


Figure  44:  (a)  Normalized  average  number  of  AP  calls  of  DFBnB  subtour  elimination,  (b)  Scaled  average 
number  of  AP  calls,  with  {(5  —  betac)  log]^o(^)>  where  /3o  =  1-49  ±  0.025. 


4.5.2  Threshing  behavior 

Figure  44(a)  shows  the  average  complexity  of  DFBnB  subtour  elimination,  measured  by  the  number  of 
calls  to  the  AP  function,  in  terms  of  the  effective  number  of  digits  for  intercity  distances.  The  result  is 
averaged  over  the  same  10,000  problem  instances  for  each  data  point  as  used  for  the  phase  transitions 
studied  in  Section  2.3.2.  Note  that  the  number  of  AP  calls  increases  significantly  from  small  problems  to 
large  ones  using  the  same  number  of  effective  digits  for  distances.  Thus,  we  normalize  the  result  in  such 
a  way  that  for  a  given  problem  size,  the  minimal  and  maximal  AP  calls  among  all  problem  instances  of 
the  same  size  are  mapped  to  zero  and  one,  respectively,  and  the  other  AP  calls  are  proportionally  adjusted 
to  a  ratio  between  0  and  1.  This  allows  us  to  compare  the  results  from  different  problem  sizes  in  one 
figure.  The  curves  in  Figure  44(a)  follow  a  paffern  similar  fo  fhaf  of  fhe  phase  fransifions  in  Section  2.3.2. 
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The  complexity  of  the  suhtour  elimination  algorithm  increases  with  the  effective  number  of  digits,  and 
exhibits  a  threshold  behavior  similar  to  phase  transitions.  Indeed,  we  can  use  finite-size  scaling  to  capture 
the  behavior  as  the  problem  size  grows,  as  illustrated  in  Figure  44(b). 

The  results  in  Figure  44  and  the  phase-transition  results  of  Section  2.3.2  indicate  that  the  complexity 
of  the  subtour  elimination  algorithm  goes  hand-in-hand  with  the  constrainedness  of  the  problem,  which  is 
determined  by  the  portion  of  distinct  entities  of  distance  matrix,  which  is  in  turn  controlled  by  the  precision 
of  distances. 

Similar  results  have  been  reported  in  [113],  where  the  effects  of  two  different  distance  distributions 
on  the  average  complexity  of  the  subtour  elimination  algorithm  were  analyzed  to  conclued  that  the  de¬ 
terminant  of  the  average  complexity  is  the  number  of  distinct  distances  of  a  problem.  The  results  of  this 
section  extend  that  in  [1 13]  to  different  sizes  of  problems  and  by  applying  finite-size  scaling  to  capture  the 
threshold  behavior  as  problem  size  increases. 

We  need  to  contrast  the  experimental  result  in  this  section  with  the  theoretical  result  on  the  NP- 
completeness  of  the  TSP  of  intercity  distances  0  and  1.  It  has  been  known  that  the  degenerated  TSP 
with  distances  0  and  1  is  still  NP-complete  [80].  On  the  other  hand,  our  experimental  results  showed  that 
when  intercity  distances  are  small,  relative  to  the  problem  size,  the  ATSP  is  easy  on  average.  Based  on  our 
experimental  result,  a  large  portion  of  the  problem  instances  with  small  intercity  distances  can  be  solved 
by  the  assignment  problem  or  Karp’s  patching  algorithm  with  no  branch-and-bound  search  required.  This 
discrepancy  indicates  that  the  worst  case  of  the  problem  is  rare  and  most  likely  pathological. 

4.6  Related  Work  and  Discussions 

Two  lines  of  previous  work  have  directly  influenced  and  inspired  fhis  research.  The  firsl  line  of  related 
work  was  on  fhe  expecfed  complexify  of  free  search,  which  shed  lighf  fo  fhe  BnB  subfour  elimination 
algorithm  described  in  Section  4.5.1  as  it  solves  the  ATSP  in  a  tree  search.  The  analysis  was  carried  out 
on  an  abstract  random  tree  model  called  incremental  tree  T  [55,  72,  73,  112,  107].  The  internal  nodes  of 
T  has  variable  number  of  children  and  edges  in  T  are  assigned  finite  and  nonnegative  random  values.  The 
cost  of  a  node  in  T  is  the  sum  of  the  edge  costs  along  the  path  from  the  root  to  that  node.  An  optimal  goal 
node  is  a  node  of  minimum  cost  at  a  fixed  depfh  d.  The  overall  goal  is  fo  find  an  opfimal  goal  node. 

There  exisf  phase  fransifions  in  fhe  cosf  of  fhe  opfimal  goal  node  and  fhe  complexify  fo  fhe  problem  of 
finding  an  opfimal  goal  in  T.  The  confrol  parameter  is  fhe  expecfed  number  of  child  nodes  of  a  common 
parenf  node  which  have  fhe  same  cosf  as  fhe  parenf.  The  cosf  of  an  optimal  goal  node  almosf  surely 
undergoes  a  phase  fransifion  from  a  linear  funclion  of  depfh  d  fo  a  consfanf  when  fhe  expecfed  same- 
cosf  children  of  a  node  increases  beyond  one.  Meanwhile,  besf-firsf  search  and  deplh-firsl  branch-and- 
bound  also  exhibif  a  phase-fransifion  behavior,  i.e.,  fheir  expecfed  complexify  changes  dramafically  from 
exponenfial  fo  polynomial  in  d  as  fhe  expecfed  same-cosf  children  of  a  node  is  reduced  fo  below  one. 
Nofe  fhaf  following  fhe  resulf  of  [21],  besf-firsf  search  is  opfimal  for  searching  fhis  random  free  among  all 
algorifhms  using  fhe  same  cosf  funclion,  in  terms  of  number  of  node  expansions,  up  fo  lie  breaking.  Thus, 
fhe  above  resulls  also  give  fhe  expecfed  complexify  of  fhe  problem  of  searching  an  incremenlal  free. 
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The  second  line  of  related  research  was  on  characterizing  the  the  assignment  problem  (AP)  lower- 
bound  cost  function  and  its  relationship  with  the  ATSP,  which  has  been  a  research  interest  for  a  long 
time  [3,  18,  30,  31,  54,  56,  74,  96].  The  first  surprising  result  [96]  is  that  the  expected  AP  cost  approaches 
a  constant  as  the  number  of  cities  n  goes  to  infinity  if  the  entries  of  distance  matrix  D  are  independent 
and  uniform  over  reals  [0, 1].  This  constant  has  been  the  subject  of  a  long  history  of  pursuit.  It  has  been 
shown  rigorously,  based  on  rigorous  replica  method  from  statistical  physics  [75],  that  the  optimal  cost  of 
random  assignment  approaches  asymptotically  to  tt^/G  [3],  which  is  approximately  1.64493.  Our  results 
in  Section  4.4  show  that  the  AP  and  the  ATSP  costs  approach  1.64421  and  1.64463,  respectively,  which 
support  the  theoretical  results  on  the  AP  cost. 

More  importantly,  the  relationship  between  the  AP  cost  and  the  ATSP  cost  has  remarkably  different 
characteristics  under  different  distance  distributions.  On  one  extreme,  the  AP  cost  is  the  same  as  the  ATSP 
cost  with  a  high  probability,  while  on  the  other  extreme,  it  can  differ  from  the  ATSP  cost,  with  a  high 
probability,  by  a  function  of  problem  size  n.  Let  AP{D)  be  the  AP  cost  and  ATSP{D)  the  ATSP  cost 
under  a  distance  matrix  D.  If  the  expected  number  of  zeros  in  a  row  of  D  approaches  infinity  when 
n  ^  oo,  then  AP{D)  =  ATSP{D)  with  a  probability  tending  to  one  [30].  However,  if  the  entities  of 
D  are  uniform  over  the  integers  [0, 1,  •  •  •  ,  [c^nj],  then  AP{D)  =  ATSP{D)  with  a  probability  going 
to  zero,  where  Cn  grows  to  infinity  with  n  [30].  Indeed,  when  the  entities  of  D  are  uniform  over  [0, 1], 
E{ATSP{D)  —  AP{D))  >  co/n,  where  cq  is  a  positive  constant  [31]. 

These  previous  results  indicate  that  the  quality  of  the  AP  function  varies  significantly,  depending  on  the 
underlying  distance  distribution.  Precisely,  the  difference  between  the  AP  cost  and  the  ATSP  cost  has  two 
phases,  controlled  by  the  number  of  zero  distances  in  the  distance  matrix  D.  In  one  phase,  the  difference 
is  zero  with  high  probability,  while  in  the  other  phase,  the  expectation  of  the  difference  is  a  function  of  the 
problem  size  n.  Our  experimental  results  in  Section  2.3.2  adds  to  this  analysis  the  existence  of  a  phase 
transition  between  these  two  phases. 

The  two-phase  result  on  the  accuracy  of  the  AP  cost  function  is  also  in  principle  consistent  with  the 
phase-transition  result  of  incremental  random  trees.  The  root  of  the  search  tree  has  a  cost  equal  to  the  AP 
cost  AP{D)  to  the  problem  and  an  optimal  goal  node  has  the  ATSP  tour  cost  ATSP{D).  If  we  subtract 
the  AP  cost  to  the  root  from  every  node  in  the  ATSP  search  tree,  the  root  node  has  cost  zero  and  an  optimal 
goal  node  has  cost  equal  to  ATSP{D)  —  AP{D).  When  there  are  a  large  number  of  zero  distances  in  D, 
there  will  be  a  large  number  of  same-cost  children,  and  the  AP  cost  of  a  child  node  in  a  search  tree  is  more 
likely  to  be  the  same  as  the  AP  cost  of  its  parent,  since  AP  will  tend  to  use  the  zero  distances.  Therefore, 
it  is  expected  that  more  nodes  in  the  search  tree  will  have  more  than  one  child  node  having  the  same  cost 
as  their  parents. 

4.7  Conclusions 

The  main  contributions  from  our  research  and  the  most  important  lessons  we  have  learned  are  twofold.  First, 
we  answered  positively  the  long-standing  question  whether  the  Traveling  Salesman  Problem  (TSP)  has 
phase  transitions  [61].  We  studied  this  issue  on  a  more  general  version  of  the  problem,  the  asymmetric 
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TSP  (ATSP).  We  empirically  showed  that  many  important  properties,  including  the  ATSP  tour  cost  and 
the  fraction  of  backbone,  have  two  characteristically  different  values,  and  the  transitions  between  them  are 
rather  abrupt  and  dramatic,  displaying  a  phase-transition  phenomenon.  The  control  parameter  of  the  phase 
transitions  is  the  effective  number  of  digits  representing  the  intercity  distances  or  the  precision  of  distance 
measure. 

Second,  our  results  provide  a  practical  guidance  on  how  to  generate  difficult  random  ATSP  problem 
instances  and  which  random  instances  should  be  used  to  compare  the  asymptotic  performance  of  ATSP 
algorithms.  A  current  common  practice  in  comparing  algorithms  when  using  a  random  ensemble  is  to 
generate  problem  instances  of  different  sizes  with  a  fixed  disfance  precision.  Our  phase  fransifion  resulfs 
indicafe  fhaf  fhe  correcf  way  is  fo  use  insfances  of  differenf  sizes  fhaf  have  fhe  same  or  similar  fealures 
such  as  fhe  same  fraclion  of  backbones.  This  in  furn  requires  increasing  fhe  precision  of  infercify  disfances 
as  fhe  problem  size  grows. 

If  is  imporfanf  fo  nofe  fhaf  fhe  exacf  locafions  of  various  phase  fransifions  presenfed  here  remain  fo  be 
malhemafically  defermined,  using  mefhods  probably  from  sfafisfical  physics  [70,  75]. 
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5  Configuration  Landscape  Analysis  and  Backbone  Guided  Local  Search 
for  Satisfiability  and  Maximum  Satisfiability 

Boolean  satisfiability  or  SAT  is  an  archetypical  decision  problem  [32].  Given  a  set  of  Boolean  variables 
and  a  set  of  clauses,  which  specify  constraints  among  the  variables,  the  problem  is  to  decide  if  there 
exists  a  variable  assignment  that  satisfies  all  consfraints.  Moreover,  there  are  overconstrained  problems 
in  the  real  world  in  which  not  all  constraints  are  satisfiable  and  the  objective  is  to  satisfy  the  maximal 
number  of  constraints.  Such  a  problem  is  called  maximum  Boolean  satisfiability  or  Max-SAT,  which  is 
the  optimization  counterpart  of  SAT.  Max-SAT  is  more  general  and  difficult  to  solve  than  SAT.  The  solution 
to  a  Max-SAT  can  be  used  to  answer  the  question  of  its  decision  counterpart.  Many  real-world  problems 
can  be  formulated  and  solved  as  SAT  or  Max-SAT,  including  scheduling,  multi-agent  cooperation  and 
coordination,  and  pattern  recognition  [6,  16,  20,  29]. 

Recent  years  have  witnessed  significant  progresses  on  SAT  in  two  directions.  The  first  is  the  under¬ 
standing  of  problem  properties,  such  as  phase  transitions  [14,  37, 48,  77]  and  backbones  in  various  difficult 
combinatorial  problems  [78, 109],  which  were  defined  in  Section  4.  It  has  been  shown  that  there  exist  sharp 
transitions  in  the  satisfiability  of  random  SAT  problem  instances  as  the  ratio  of  the  number  of  clauses  to 
the  number  of  variables  (clause/variable  or  CA^  ratio)  increases  beyond  a  critical  value  [14,  77],  a  phe¬ 
nomenon  similar  to  phase  transitions  in  disordered  systems  [37,  44,  48].  It  has  also  been  observed  that  the 
fraction  of  backbone  variables,  the  ones  that  have  fixed  values  among  all  optimal  solutions  to  Max-SAT, 
increases  abruptly  as  the  clause/variable  ratio  goes  across  a  critical  value,  yet  another  phenomenon  similar 
to  phase  transitions  [109]. 

The  second  research  direction  is  focused  on  developing  efficient  SAT  solvers,  especially  local  search 
algorithms  [71,  89,  87].  The  best  known  local  search  algorithms  include  WalkSAT  [87]  and  its  varia¬ 
tions  [71].  These  algorithms  are  able  to  significantly  outperform  systematic  search  algorithms  on  most 
random  problem  instances  and  some  problem  classes  of  real  applications,  solving  larger  satisfiable  prob¬ 
lem  insfances  in  less  time,  albeit  the  former  may  fail  to  reach  a  solution  on  a  particular  instance  even  if  such 
a  solution  exists.  The  great  success  of  WalkSAT  has  subsequently  led  to  the  paradigm  of  formulating  and 
solving  difficult  problems  from  other  problem  domains,  such  as  planning,  as  satisfiability  problems  [57]. 
The  SAT-based  approach  is  now  among  the  best  methods  for  planning. 

A  problem  of  fundamental  interest  and  practical  importance  is  how  to  utilize  problem  structural  in¬ 
formation,  such  as  that  of  phase  transitions  and  backbones,  in  a  search  algorithm  to  cope  with  the  high 
computational  cost  of  difficult  problems,  as  well  as  to  improve  the  performance  of  the  algorithm.  The 
published  work  on  this  topic  is  limited.  [84]  developed  a  transformation  method  that  exploits  phase  tran¬ 
sitions  of  tree  search  problems,  and  [22]  proposed  a  method  to  incorporate  backbone  information  from 
SAT  in  a  systematic  search  algorithm.  [95]  proposed  a  heuristic  backbone  sampling  method  for  generating 
initial  assignments  for  a  local  search  for  Max-SAT.  Despite  the  success  of  these  previous  works,  much 
research  needs  to  be  carried  out  on  exploiting  structural  information  in  order  to  demonstrate  the  viability 
of  incorporating  such  information  in  search  algorithms. 

One  of  the  challenges  in  utilizing  the  structural  information  of  a  problem,  such  as  phase  transitions  or 
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backbones,  in  a  search  algorithm,  is  to  make  the  algorithm  not  only  work  on  random  problem  instances, 
but  also  perform  well  on  individual  problem  instances,  especially  those  from  real-world  applications.  To 
our  knowledge  no  result  on  individual  real  problem  instances  has  been  reported  in  the  previous  work  using 
structural  information.  A  phase-transition  property  is  a  characteristic  of  a  collection  of  problem  instances 
drawn  from  a  common  distribution.  Therefore,  phase-transition  information  may  only  help  when  solving 
a  set  of  related  problem  instances  from  which  structural  information  can  be  extracted.  For  individual 
problem  instances,  however,  information  of  an  ensemble  may  be  irrelevant.  Therefore,  new  structural 
information  must  be  acquired  and  new  mechanism  of  applying  such  information  must  be  designed. 

In  this  section  we  are  mainly  interested  in  Max-SAT,  thanks  to  its  generality  and  broad  applicability  in 
practice,  and  local  search  algorithms,  the  WalkS  AT  algorithm  in  particular,  which  was  originally  developed 
for  SAT  problem  instances.  We  first  investigate  the  configuration  landscapes  of  local  minima  reached  by 
WalkSAT.  Briefly,  we  define  the  configuration  landscape  of  a  set  of  local  minima  as  their  distribution 
with  respect  to  their  cost  and  structure  differences  in  reference  to  all  optimal  solutions  or  a  particular 
local  minimum.  Similar  global  structures  have  been  analyzed  on  other  optimization  problems,  such  as 
the  Traveling  Salesman  problem  (TSP)  and  graph  bisection  problem,  and  local  search  algorithms,  such 
as  2-opt  for  the  TSP  [9,  61].  Our  experimental  results  on  SAT  and  Max-SAT  show  that  local  minima 
from  WalkSAT  reside  close  to  one  another,  forming  clusters  in  configuration  landscapes.  The  results  also 
indicate  that  WalkSAT  is  also  effective  for  Max-SAT,  finding  many  high  quality  local  minima  that  are  very 
close  to  optimal  solutions  in  terms  of  cost  and  structure  differences.  Although  we  carry  out  this  analysis  in 
part  for  the  purpose  of  developing  a  new  method,  the  work  itself  and  the  results  are  of  interest  of  their  own. 
Our  results  indicate  that  WalkSAT  is  not  only  an  efficient  algorithm  for  SAT,  but  also  a  good  choice  for 
Max-SAT.  To  our  knowledge,  this  study  appears  to  provide  the  first  systematic  performance  analysis  on 
WalkSAT  for  Max-SAT.  Previous  research  also  showed  that  WalkSAT  is  an  effective  method  for  solving 
overconstrained  Steiner  Tree  problems  [50]. 

The  second  main  contribution  of  this  section  is  an  innovative  and  general  heuristic  method  that  ex¬ 
ploits  backbone  information  to  improve  the  performance  of  a  local  search  algorithm.  The  new  method  is 
driven  by  the  results  on  the  configuration  space  analysis  of  local  minima  uncovered  in  the  first  part  of  this 
section.  It  is  also  inspired  by  the  previous  research  on  phase  transitions  [14,  37,  48,  77]  and  backbones 
of  combinatorial  problems  [78,  109].  This  method  is  built  upon  the  following  working  hypothesis:  On  a 
problem  whose  optimal  and  near  optimal  solutions  form  a  cluster,  if  a  local  search  algorithm  can  reach 
close  vicinities  of  such  solutions,  the  algorithm  will  be  effective  in  finding  some  information  of  the  solution 
structures,  backbone  in  particular.  This  implies  that  the  local  minima  reached  by  the  algorithm  must  share 
many  parts  of  the  solution  structures  with  the  optimal  solutions.  If  we  extract  such  structure  information 
from  local  minima,  we  can  then  use  it  to  adjust  the  local  search  in  such  a  way  that  it  moves  directly  toward 
the  regions  of  the  search  space  containing  high  quality  solutions.  Using  Max-SAT  and  the  WalkSAT  local 
search  algorithm,  we  demonstrate  how  this  new  method  can  improve  WalkS  AT’s  effectiveness.  We  empir¬ 
ically  show  that  the  new  method  is  effective  on  random  problem  instances  as  well  as  real  problem  instances 
from  a  SAT  library  (SATLIB  [47]),  increasing  WalkSAT’s  probability  of  reaching  better  solutions. 
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We  proceed  as  follows  in  the  rest  of  the  section.  In  Section  5. 1,  we  first  describe  SAT  and  Max-SAT,  as 
well  as  the  WalkSAT  local  search  algorithm  for  both  SAT  and  Max-SAT.  In  this  section,  we  also  describe 
an  extension  to  WalkSAT  that  allows  a  dynamic  parameter  tuning  [46]  at  runtime,  which  we  utilize  to  free 
WalkSAT  from  reliance  on  a  manually  set  noise  parameter.  We  investigate  the  configuration  landscapes  of 
local  minima  from  WalkSAT  in  Section  5.2.  We  then  develop  the  backbone  guided  local  search  algorithm 
in  Section  5.3.  We  discuss  the  main  idea  of  this  method,  consider  how  it  can  be  incorporated  in  WalkSAT  to 
make  biased  moves,  and  describe  ways  of  capturing  backbone  information.  We  then  present  in  Section  5.4 
experimental  results  of  backbone  guided  local  search  on  random  problem  instances  and  instances  from 
SATLIB  [47].  We  conclude  in  Section  5.5  with  some  discussions  for  future  work. 

An  early  version  of  this  section  appeared  in  [1 15].  The  software  we  developed  and  used  in  this  research 
is  freely  available  at  http://www.cse.wustl.edu/~zhang/projects/bgwalksat/index.html. 

5.1  SAT,  Max-SAT,  and  WalkSAT  local  search 

We  provide  in  this  section  some  background  information  on  SAT  and  Max-SAT  problems  and  the  WalkSAT 
local  search  algorithm  for  SAT  [71,  87].  We  also  review  an  existing  refinement  to  WalkSAT  for  SAT  [46], 
which  eliminates  WalkSAT’s  dependence  on  a  manually  set  noise  parameter,  and  demonstrate  its  efficacy 
for  Max-SAT. 

5.1.1  Boolean  satisfiability  and  maximum  satisfiability 

A  Boolean  satisfiability,  or  SAT,  is  typically  represented  by  a  formula  consisting  of  a  set  of  Boolean 
variables  and  a  conjunction  of  a  set  of  disjunctive  clauses  of  literals,  which  are  variables  or  their  negations. 
A  clause  is  satisfied  if  one  of  its  literals  is  set  to  true,  and  a  formula  is  satisfied  if  no  clause  is  violated. 
A  formula  defines  constraints  on  the  possible  combinations  of  variable  assignments  in  order  to  satisfy  the 
formula.  A  SAT  problem  is  to  decide  if  a  variable  assignment  exists  that  satisfies  all  the  clauses.  When  not 
all  clauses  can  be  satisfied,  the  objective  is  to  maximize  the  number  of  satisfied  clauses  and  the  problem 
becomes  maximum  Boolean  satisfiability,  or  Max-SAT,  which  is  an  optimization  problem.  A  SAT  or 
Max-SAT  with  k  literals  per  clause  is  short  handed  as  /c-SAT  or  Max-/c-SAT,  respectively.  It  is  known 
that  fc-SAT  with  k  being  at  least  three  is  NP-complete  and  Max-/c-SAT  with  k  at  least  two  is  NP-hard  [32], 
meaning  that  there  is  no  known  polynomial  algorithm  for  the  problems,  and  it  is  unlikely  such  an  algorithm 
exists. 

5.1.2  The  WalkSAT  local  search  algorithm 

Since  the  WalkSAT  local  search  algorithm  [87]  was  developed  to  solve  Boolean  satisfiability,  the  existing 
study  of  WalkSAT  and  its  variations  has  mainly  concentrated  on  satisfiable  SAT  instances.  As  Max-SAT, 
which  includes  satisfiable  as  well  as  unsatisfiable  instances,  is  our  main  focus  in  this  research,  we  are 
interested  in  the  effectiveness  of  WalkSAT  in  finding  optimal  solutions  to  both  satisfiable  and  unsatisfiable 
instances. 
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Figure  45:  Main  operations  in  a  try  of  WalkSAT. 


WalkSAT  is  a  randomized  algorithm.  The  algorithm  and  its  variations  all  follow  the  same  overall 
procedure  that  starts  with  an  initial  random  variable  assignment  and  makes  moves  by  flipping  one  variable 
at  a  time  from  True  to  False  or  vice  versa,  until  it  finds  a  satisfying  assignment  or  reaches  a  predefined 
maximal  number  of  flips.  Each  such  attempt  is  called  a  tty  or  restart.  The  procedure  repeats  until  a 
maximal  number  of  tries  has  been  attempted. 

To  select  which  variable  to  flip  in  each  step,  the  effect  of  flipping  a  variable  is  assessed.  Flipping  a 
variable  may  make  some  unsatisfied  clauses  satisfied,  and  some  satisfied  clauses  unsatisfied.  The  numbers 
of  clauses  that  will  be  made  unsatisfied  by  flipping  a  variable  is  called  the  break-count  of  the  variable  at 
the  current  assignment.  WalkSAT  attempts  to  flip  a  variable  with  zero  break-count,  trying  to  make  the 
next  assignment  no  worse  than  the  current  one.  To  find  such  a  variable  with  zero  break-count,  WalkSAT 
first  selects  an  unsatisfied  clause  C,  uniformly  randomly,  from  all  unsatisfied  clauses.  This  is  called  clause 
pick.  If  C  has  a  variable  of  zero  break-count,  WalkSAT  then  picks  such  a  variable,  uniformly  randomly, 
from  the  ones  that  qualify  (called  flat  pick).  If  no  zero  break-count  variable  exists  in  C,  WalkSAT  then 
makes  a  random  choice.  With  probability  p  it  chooses,  uniformly  randomly,  a  variable  from  all  the  variables 
involved  in  C  (called  noise  pick);  or  with  probability  1  —  p  it  selects  a  variable  with  the  least  break-count, 
breaking  a  tie  arbitrarily  if  multiple  choices  exist  (called  greedy  pick).  Figure  1.  The  algorithm  takes 
three  Figure  45.  The  algorithm  takes  three  parameters  to  run:  the  number  of  tries,  the  maximal  number  of 
flips  in  each  try,  and  a  probability  for  noise  pick,  which  is  commonly  referred  to  as  the  noise  ratio  of  the 
algorithm. 


72 


5.1.3  WalkSAT  with  dynamic  noise  strategy 

One  limitation  of  the  WalkSAT  family  of  algorithms  is  their  dependenee  on  a  manually  set  noise  ratio. 
To  be  effeetive,  the  noise  ratio  needs  to  be  tuned  for  eaeh  individual  problem,  espeeially  for  those  that  do 
not  share  eommon  features.  So  far,  two  methods  have  been  proposed  to  resolve  this  issue  for  SAT.  Auto- 
WalkSAT  [81]  uses  a  probing  phase  to  estimate  the  optimal  parameter  for  the  noise  ratio.  The  estimated 
noise  ratio  is  then  adopted  throughout  the  seareh  phase  of  the  algorithm.  Similar  to  the  original  WalkSAT, 
An  to- Walks  AT  uses  a  statie  noise  ratio. 

Deviating  from  the  statie  strategy,  WalkSAT  with  dynamie  noise  [46]  adopts  the  strategy  of  automat- 
ieally  adjusting  noise  ratio  as  the  seareh  progresses.  In  other  words,  the  dynamie  strategy  uses  different 
noise  ratios  at  different  stages  of  the  seareh.  This  strategy  seems  to  be  more  reasonable  than  the  statie 
strategy.  It  is  relatively  easier  to  make  great  progress  at  an  early  stage  of  a  loeal  seareh  than  at  a  later  stage, 
therefore  the  noise  ratio  should  be  adjusted  aeeordingly,  depending  on  where  the  eurrent  seareh  is  in  the 
overall  seareh  spaee. 

The  idea  of  dynamie  noise  strategy  is  simple:  start  a  loeal  seareh  with  the  noise  ratio  equal  to  zero,  and 
examine  the  number  of  violations  in  the  eurrent  state  every  9m  flips,  where  m  is  the  number  of  elauses 
of  a  given  problem,  and  9  a  eonstant.  If  the  number  of  violations  has  not  deereased  sinee  the  last  time 
we  eheeked  {9m  flips  ago),  the  seareh  is  assumed  to  have  stagnated,  and  the  noise  ratio  is  inereased  to 
wp  +  (1  —  wp)(j),  where  wp  is  the  eurrent  noise  ratio  and  (p  is  another  eonstant.  Otherwise,  the  noise  ratio 
is  deereased  to  wp{l  —  2(1)).  The  diserepaney  between  the  formulas  for  inereasing  and  deereasing  the  noise 
ratio  is  based  on  some  empirieal  observations  of  how  WalkSAT  behaves  when  the  noise  ratio  is  too  high, 
eompared  with  how  it  behaves  when  the  parameter  is  too  low  [46].  The  dynamie  strategy  was  designed 
and  tested  with  WalkSAT’s  eutoff  parameter  set  to  infinity;  i.e.,  no  random  restarts.  This  is  the  setting  we 
use  for  WalkSAT  with  dynamie  noise  for  all  of  our  experiments  for  SAT  and  Max-SAT.  For  eonvenienee, 
we  refer  to  this  strategy  as  Dyna-WalkSAT  in  the  remaining  of  the  seetion. 

Note  that  Dyna-WalkSAT  uses  two  parameters,  9  and  cp.  The  differenee  of  using  these  two  new  pa¬ 
rameters  and  using  the  noise  ratio  in  the  original  algorithm  is  that  these  two  parameters  do  not  have  to  be 
tuned  for  every  single  problem  instanee;  the  performanee  of  Dyna-WalkSAT  with  the  same  values  for  9 
and  (p  is  relatively  eonsistent  aeross  different  problem  instanees.  Following  [46],  we  have  set  0  =  1  /6  and 
p  =  1/5  in  our  experiments  for  SAT  and  Max-SAT. 

Though  Dyna-WalkSAT  was  originally  designed  and  tested  within  the  eontexts  of  SAT,  we  have  found 
it  to  be  effeetive  on  Max-SAT  as  well.  In  our  experiments,  we  used  problem  instanees  of  2,000  variables 
and  CfV  ratios  of  4.3,  6.0  and  8.0  to  eapture  problem  instanees  from  different  eonstrainedness  regions. 
We  generated  1,000  random  problem  instanees  at  eaeh  of  these  C/V  ratios.  The  problem  instanees  were 
random  in  that  a  elause  was  generated  by  uniformly  pieking  three  literals,  without  replaeement,  and  by 
disearding  duplieate  elauses.  For  WalkSAT,  the  noise  ratios  were  set  from  0.0  to  0.9,  with  an  inerement  of 
0.1,  the  number  of  tries  per  problem  instanee  at  100,  and  the  number  of  flips  per  try  at  10,000.  We  also  ran 
Dyna-WalkSAT  on  eaeh  of  these  problem  instanees.  To  make  a  fair  eomparison,  we  let  Dyna-WalkSAT 
exeeute  one  million  flips  total.  Dyna-WalkSAT  also  used  the  same  parameters  as  used  by  WalkSAT,  exeept 
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Figure  46:  Experimental  validation  of  Dyna-WalkS AT  on  random  Max-3-SAT,  for  2,000- variable  problem 
instances. 


the  noise  ratio.  To  reiterate,  following  [46]  we  set  0  =  1/6  and  (j)  =  1/5  in  Dyna-WalkSAT,  which  have 
been  found  to  be  effective  over  a  wide  range  of  SAT  and  Max-SAT  instances. 

The  experimental  results  are  shown  in  Figure  2.  The  horizontal  axes  are  noise  ratios  for  WalkSAT 
and  the  vertical  axes  record  the  average  solution  quality.  The  error  bars  in  the  figures  measure  the  95% 
confidence  infervals  of  fhe  resulfs.  For  all  fhree  C/V  rafios  lesled,  fhe  performance  of  Dyna- WalkSAT  is 
very  close  fo  fhe  performance  of  WalkSAT  wifh  fhe  opfimal  noise  rafio,  indicafing  fhaf  fhe  dynamic  noise 
sfrafegy  is  effeclive  for  3-SAT  and  Max-3-SAT. 

Due  fo  ifs  simplicity  and  reasonable  performance,  in  fhe  resf  of  fhis  secfion  we  will  use  Dyna- WalkSAT 
wifh  0=1/6  and  </  =  1/5  as  defaull  parameters  fo  replace  WalkSAT  in  our  experimenfal  analysis. 

5,2  Configuration  Landscapes 

Given  fwo  variable  assignmenfs  fo  a  given  SAT  or  Max-SAT  problem  insfance,  we  can  measure  fheir 
differences  in  fwo  ways.  The  firsl  is  fhe  difference  of  fheir  cosfs  or  fhe  numbers  of  violated  clauses.  This 
difference  can  be  normalized  (divided)  by  fhe  fofal  number  of  clauses,  giving  fhe  difference  of  violafions 
per  clause.  The  second  quanfify  measures  sfrucfural  difference  in  fhe  form  of  fhe  Hamming  disfance 
befween  fhe  fwo  assignmenfs.  Since  a  solufion  fo  a  SAT  problem  is  simply  a  siring  of  0  and  1,  Hamming 
disfance  here  is  simply  fhe  convenlional  Hamming  disfance  for  binary  slrings.  The  Hamming  disfance  can 


74 


also  be  normalized  by  the  total  number  of  variables,  resulting  in  the  normalized  Hamming  distanee  per 
variable.  We  adopt  normalized  eost  differenee  and  normalized  Hamming  distanee  to  make  the  results  from 
problems  of  different  sizes  direetly  eomparable. 

With  the  relative  solution  quality  and  strueture  differenee  of  two  assignments  speeified,  we  define  the 
eonfiguration  landseape  of  a  set  of  assignments  or  solutions  as  the  distribution  of  the  solutions  in  terms  of 
their  qualities  and  strueture  differenees  relative  to  a  referenee  solution,  whieh  ean  be  an  optimal  solution 
or  the  best  solution  in  a  given  set.  The  set  of  solutions  ean  be  all  the  optimal  solutions,  all  suboptimal 
solutions  up  to  a  fixed  bound,  as  well  as  loeal  minima  from  a  loeal  seareh. 

We  ean  use  landseape  eonfiguration  to  eapture  the  effeetiveness  of  the  WalkSAT  and  Dyna-WalkSAT 
algorithms  on  3-SAT  and  MAX-3-SAT.  We  earried  out  two  sets  of  experiments.  In  the  first  set  of  exper¬ 
iments,  we  aimed  to  direetly  measure  the  effeetiveness  of  WalkSAT  in  terms  of  finding  optimal  and  near 
optimal  solutions.  To  this  end,  we  used  all  optimal  solutions  to  measure  the  quality  of  a  set  of  loeal  minima 
from  WalkSAT.  Sinee  finding  all  optimal  solutions  is  eomputationally  expensive,  we  restrieted  ourselves 
to  relatively  small  random  problem  instanees  with  100  variables  and  CfV  ratios  of  2.0,  4.3,  6.0  and  8.0 
to  eapture  problems  in  different  eonstrainedness  regions.  We  generated  1,000  problems  for  eaeh  C/V  ra¬ 
tio.  The  problems  were  randomly  generated  by  uniformly  pieking  three  literals  without  replaeement  for  a 
elause,  with  duplieate  elauses  disearded. 

To  find  all  opfimal  assignments  to  a  Max-SAT  problem,  we  extended  the  well  known  Davis-Putnam- 
Logemann-Loveland  (DPLL)  algorithm  for  SAT  [19]  to  Max-SAT.  We  ran  our  extended  DPLL  algorithm 
for  Max-SAT  [102]  and  WalkSAT  on  the  same  set  of  100-variable  Max-3-SAT  problem  instanees.  For 
WalkSAT,  we  set  the  number  of  tries  per  problem  at  100,  the  number  of  flips  per  try  at  10,000,  and 
the  noise  ratio  at  0.5.  We  then  examined  the  eonfiguration  landseapes  of  the  loeal  minima  reaehed  by 
WalkSAT  against  the  optimal  solutions  in  terms  of  the  eost  differenee  between  a  loeal  minimum  and  an 
optimal  solution  as  well  as  the  Hamming  distanee  of  the  loeal  minimum  to  its  nearest  optimal  solution. 
Note  that  the  Hamming  distanee  of  a  loeal  minimum  in  faet  measures  the  minimal  number  of  flips  required 
to  turn  the  loeal  minimum  into  an  optimal  solution. 

The  eonfiguration  landseapes  of  loeal  minima  from  WalkSAT  are  summarized  in  Figure  47.  Sinee 
WalkSAT  is  very  effieient  on  undereonstrained  SAT  instanees,  finding  satisfiable  solutions  on  nearly  all 
problem  instanees  when  the  CfV  ratio  is  2.0,  we  do  not  inelude  the  results  for  a  C/V  ratio  of  2.0  here. 
The  X-Y  planes  in  the  figures  show  the  eorrelation  between  the  normalized  Hamming  distanee  and  the 
normalized  eost  differenee.  Eaeh  point  on  the  X-Y  plane  represents  a  set  of  possible  loeal  minima  with  the 
same  eost  differenee  and  Hamming  distanee  that  may  be  visited  by  WalkSAT.  The  origins  of  the  figures 
eorrespond  to  global  optima.  The  vertieal  Z  axes  measure  the  total  numbers  of  loeal  minima  reaehed  by 
WalkSAT. 

As  shown  in  Figure  47(a),  WalkSAT  performs  well  on  undereonstrained  and  eritieally  eonstrained 
problems,  in  that  it  ean  find  global  minima  very  often.  This  is  shown  by  the  point  on  the  Z  axis  indieated  by 
the  arrow  in  the  figure.  Therefore,  WalkSAT  is  effeetive  in  finding  optimal  solutions  on  undereonstrained 
and  eritieally  eonstrained  problems.  However,  the  number  of  loeal  minima  that  are  also  global  optima 
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(a)  v=1 00,  cv=4.3  00-Cv=6.0 


Figure  47:  Configuration  landscapes  of  local  minima  from  WalkSAT  on  100  variable  random  3-SAT  and 
Max-3-SAT,  relative  to  optimal  solutions. 


decreases  from  66,677  to  8,616  as  the  CA^  ratio  increases  from  2.0  to  4.3,  indicating  that  the  effectiveness 
of  WalkSAT  decreases.  This  number  decreases  further  from  201  to  0  on  overconstrained  problems  with 
CfV  ratios  of  6.0  and  8.0,  respectively  (Figures  47(b)  and  (c)).  This  result  indicates  that  WalkSAT  becomes 
less  effective  at  finding  optimal  solutions  as  problem  constrainedness  increases.  Figure  48  shows  the 
contours  of  the  configuration  landscapes  in  Figures  47(b)  and  (c)  on  the  X-Y  planes,  showing  a  nearly 
linear  correlation  between  the  cost  difference  and  the  Hamming  distance.  That  is,  a  local  minimum  that 
has  a  high  cost  tends  to  have  a  large  Hamming  distance  to  an  optimal  assignment. 

In  the  second  set  of  experiments,  we  examined  the  configuration  landscapes  of  local  minima  from 
Dyna- WalkSAT.  We  used  2,000-variable  random  3-SAT  and  Max-3-SAT  with  CfV  ratios  of  4.3,  6.0  and 
8.0.  As  before,  we  generated  1,000  problem  instances  for  each  C/V  ratio.  Because  the  problems  were  too 
large  to  be  solved  optimally,  we  built  a  configuration  landscape  of  a  set  of  local  minima  with  respect  to  the 
best  local  minimum  among  them.  The  results  are  shown  in  Figure  49,  which  are  qualitatively  similar  to 
the  configuration  landscapes  in  Figure  47. 

An  interesting  result  from  these  experiments  is  that  the  configuration  landscapes  of  local  minima 
reached  by  WalkSAT  and  Dyna-WalkSAT  exhibit  bell  surfaces  on  overconstrained  problems  with  large 
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normalized  Hamming  distance 

Figure  48:  Contours  of  the  configuration  landscapes  of  local  minima  from  WalkSAT  on  100  variable  Max- 
SAT  with  C/V  ratios  of  6.0  and  8.0. 


CA^  ratios.  More  importantly,  the  summit  of  such  a  bell  surface  shifts  away  from  optimal  solutions,  the 
(0,0)  point  on  the  X-Y  plane,  as  the  C/V  ratio  increases.  This  observation  is  examplified  by  the  contours  in 
Figure  48.  Nevertheless,  despite  the  increased  difficult  of  Max-SAT  as  the  C/V  ratio  grows,  WalkSAT  and 
Dyna- WalkSAT  are  still  fairly  effective  in  that  they  are  able  to  reach  local  minima  that  are  close  to  global 
optima.  For  100-variable  instances  with  C/V  ratio  of  8.0  (Figure  48(b)),  the  majority  of  local  minima 
reached  by  WalkSAT,  the  ones  located  at  the  peak  point  of  the  bell  surface  of  the  figure,  have  a  normalized 
cosf  difference  fo  opfimal  solufions  of  0.014  for  100  variable  problems,  which  is  equivalenf  fo  abouf  eleven 
more  consfrainfs  violafed  fhan  an  opfimal  solution  on  such  overly  consfrained  problem  insfances.  Since 
WalkSAT  is  typically  execufed  wifh  mulfiple  frials,  fhe  besf  local  minimum  if  can  landed  on  will  be  much 
heifer  fhan  such  mosl  likely  local  minima. 

Anofher  inferesfing  and  imporfanf  observafion  of  fhe  resulfs  in  Figure  48  is  fhaf  fhere  is  a  near  linear 
correlation  belween  fhe  cosf  difference  belween  a  local  minimum  and  ifs  nearesl  optimal  solufion  and  fheir 
Hamming  disfance,  fhe  fwo  qualify  measuremenfs  we  adopf.  This  is  evidenl  fhaf  fhe  contours  in  Figure  48 
have  ellipse  shapes.  An  implicafion  of  fhis  observafion  is  fhaf  a  local  minimum  wifh  a  small  cosf  is  more 
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(a)  c/v=4.3 


(b)  c/v=6 


(c)  c/v=8 


Figure  49:  Local  minima  from  WalkSAT  on  2, 000- variable  Max-3-SAT  with  C/V  ratios  of  4.3,  6  and  8. 


likely  to  share  a  larger  common  solution  structure  with  an  optimal  solution.  More  importantly,  majority 
local  minima  have  most  parts  of  their  variable  assignments  consistent  with  their  nearest  global  optima. 
For  instance,  the  majority  local  minima  on  100-variable  problems  with  C/V  ratio  of  8.0  have  normalized 
Hamming  distances  around  0.13.  This  means  that  out  of  100  variables,  87  of  them  are  correctly  set.  This 
implies  that  these  local  minima  must  share  large  portions  of  the  variable  assignments  with  the  optimal 
solutions.  We  will  exploit  this  phenomenon  in  our  new  search  algorithm  in  the  next  section. 

In  concluding  this  section,  we  need  to  point  out  that  the  results  of  this  section  extended  the  previous 
studies  on  global  structures  of  optimization  cost  surfaces  [8,  9,  61].  It  has  been  shown  in  these  studies  (and 
other  works  cited  in  [8,  9,  61])  that  there  exists  a  correlation  between  the  cost  differences  and  distances 
among  local  minima  of  such  optimization  problems  as  the  symmetric  Traveling  Salesman  problem  and 
graph  bisection  problems.  [8,  9]  particularly  showed  a  near  linear  relation  between  the  quality  of  local 
minima  and  distances  among  them,  which  is  similar  to  the  contour  results  in  Figure  48.  Our  results  made 
two  noticeable  extensions  to  the  study  of  global  structures  of  optimization  problems.  First,  we  consid¬ 
ered  configuration  landscape  defined  as  distribution  of  local  minima  with  respect  to  the  cost  differences 
and  distances  between  the  local  minima  and  optimal  solutions  or  best  local  minima.  This  allows  us  to 
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directly  examine  globally  convex  property  or  “big  valley”  structure  of  the  cost  surfaces  of  optimization 
problems  [9].  Secondly,  our  results  showed  that  there  indeed  exists  a  big  valley  structure  in  the  con¬ 
figuration  landscapes  of  3-SAT  and  Max-3-SAT,  supporting  the  “big  valley”  conjecture  on  optimization 
problems  [8]. 

Furthermore,  we  would  also  like  to  point  out  that  the  “big  valley”  results  in  [8,  9]  led  to  a  new  multi¬ 
start  strategy  that  utilize  such  a  “big  valley”  structure  of  local  minima.  In  this  section,  we  present  a 
different,  novel  way  to  exploit  such  global  structures,  which  is  the  topic  of  the  next  section. 

5.3  Backbone  Guided  Local  Search 

In  this  section,  we  discuss  in  detail  the  backbone  guided  local  search  algorithm.  We  first  present  the  main 
idea  and  then  discuss  how  it  can  be  applied  to  WalkS  AT,  forming  the  backbone  guided  WalkSAT  algorithm. 
We  also  consider  two  different  ways  to  estimate  backbone  frequencies  using  local  minima. 

5.3.1  Main  ideas 

The  backbone  variables  of  a  problem  are  the  ones  that  are  critically  constrained,  since  they  must  be  set 
to  particular  values  to  make  an  optimal  solution  feasible.  By  the  same  token,  if  a  pair  of  a  variable  and 
one  of  its  values  appears  more  often  in  the  set  of  all  optimal  solutions,  the  variable  is  more  constrained. 
If,  somehow,  we  knew  the  frequency  of  a  variable- value  pair  in  all  optimal  solutions,  we  could  construct 
a  “smart”  search  algorithm  by  using  the  backbone  frequency  information  as  an  oracle  to  guide  each  step 
of  the  algorithm.  Take  WalkSAT  as  an  example.  At  each  step  of  the  algorithm,  we  can  use  the  backbone 
frequencies  to  change  the  way  in  which  a  variable  is  chosen  to  flip,  i.e.,  we  prefer  flipping  a  variable  fhaf 
is  unsynchronized  wifh  ifs  backbone  frequency  more  fhan  anofher  variable  under  fhe  currenf  assignmenf. 
In  ofher  words,  we  should  focus  on  fixing  fhe  critically  consfrained  variables  fhaf  are  nol  currenfly  sef 
correcfly. 

Unforlunalely,  exacf  backbone  frequencies  of  a  problem  are  even  more  difficull  fo  come  by  fhan  acfual 
problem  solufions.  To  address  fhis  problem,  fhe  second  key  idea  of  backbone  guided  local  search  is  fo 
esfimafe  backbone  frequencies  using  local  minima  of  a  local  search.  We  simply  freaf  local  minima  as  if 
fhey  were  optimal  solufions  and  compufe  pseudo  backbone  frequencies,  which  are  an  esfimafion  of  real 
backbone  frequencies.  More  precisely,  we  define  fhe  pseudo  backbone  frequency  of  a  liferal  (a  variable- 
value  pair)  as  fhe  frequency  wifh  which  fhe  liferal  appears  in  all  local  minima,  which  we  denote  as  p{l) 
where  I  is  a  literal.  Note  fhaf  p{l)  =  1  —  where  ->1  is  fhe  negation  of  1. 

The  qualify  of  pseudo  backbone  frequencies  depends  on  fhe  effectiveness  and  efficiency  of  fhe  local 
search  algorifhm  used.  As  discussed  in  Secfion  5.2,  high-qualify  local  minima  can  be  obfained  by  efficienl 
local  search  algorifhms,  such  as  WalkSAT.  Even  fhough  WalkSAT  may  land  on  subopfimal  solution  wifh 
fairly  high  probabilities,  particularly  on  overconsfrained  problem  insfances,  mosf  of  fhe  local  minima  from 
WalkSAT  indeed  have  large  porfions  of  variables  sef  fo  fhe  correcf  values,  so  fhaf  fhey  confain  parfs  of 
opfimal  solufions  or  parfial  backbone.  In  fhis  research,  we  adopf  WalkSAT  fo  collecf  local  minima  and 
fhen  in  refurn  apply  fhe  backbone  guided  search  mefhod  fo  WalkSAT  fo  improve  ifs  performance. 
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5.3.2  Biased  moves  and  selections 


Pseudo  backbone  frequencies  can  be  incorporated  in  a  local  search  algorithm  to  make  ’’biased”  moves  or 
flips.  Consider  a  simple  example  of  two  variables,  xi  and  X2,  that  appear  in  a  violated  clause  and  have  the 
same  effect  under  the  current  assignment,  i.e.,  flipping  one  of  them  makes  the  violated  clause  satisfied,  and 
both  variables  have  the  same  break-count  or  will  cause  the  same  number  of  satisfied  clauses  unsatisfied  if 
flipped.  Let  B  be  the  set  of  backbone  variables  along  with  their  fixed  values,  T  be  the  set  of  local  minima 
from  which  pseudo  backbone  frequencies  were  computed,  and  vi  and  V2  are  the  current  values  of  xi  and 
X2-  We  will  prefer  to  flip  xi  over  X2  if  under  the  current  assignment,  P{{xi  =  vi)^B\T}  <  P{{x2  = 
V2)^B\T},  which  means  that  under  the  current  assignment,  xi  is  less  likely  to  be  part  of  backbone  than 
X2,  given  the  set  of  local  minima  T.  Note  that  P{{x  =  v)^B\P}  is  the  pseudo  backbone  frequency  of 
literal  x  =  v  under  the  evidence  of  a  set  of  local  minima  T. 

How  can  the  pseudo  backbone  frequencies  be  used  to  alter  the  way  that  WalkSAT  chooses  variables? 
As  discussed  in  Section  5.1.2,  WalkSAT  makes  uniformly  random  choices  in  selecting  a  variable  to  flip 
when  multiple  choices  exist.  For  example,  when  there  are  more  than  one  unsatisfied  clause  under  the 
current  assignment,  WalkSAT  arbitrarily  (uniformly)  chooses  an  unsatisfied  clause.  Similarly,  when  there 
are  multiple  variables  with  zero  break-count,  WalkSAT  chooses  one  arbitrarily. 

Based  on  the  maximum  entropy  principle  [49],  it  is  optimal  on  average  to  make  an  unbiased  choice  if 
there  is  no  information  to  distinguish  one  choice  over  another.  Therefore,  with  no  additional  information 
on  optimal  assignments,  the  WalkSAT  algorithm  is  optimal  on  average  in  terms  of  selecting  a  variable  to 
flip.  In  backbone  guided  search,  we  apply  pseudo  backbone  information  to  force  WalkSAT  to  make  random 
but  biased  choices.  If  a  backbone  variable  and  a  nonbackbone  variable  can  make  a  clause  satisfied,  the 
backbone  variable  should  be  chosen.  In  other  words,  we  modify  WalkSAT’s  random  strategies  in  such  a 
way  that  a  backbone  or  overconstrained  variable  will  be  chosen  more  often  than  a  nonbackbone  variable. 
To  this  end,  we  use  pseudo  backbone  frequencies  to  help  make  random  biased  selections. 

5.3.3  Backbone  guided  WalkSAT 

Backbone  guided  WalkSAT  has  two  phases.  The  first  is  a  estimation  phase  that  collects  local  minima  by 
running  WalkSAT,  with  a  fixed  number  of  tries.  The  local  minima  thus  collected  are  compiled  to  compute 
the  pseudo  backbone  frequencies  of  all  literals. 

The  second  phase  carries  out  the  actual  backbone  guided  local  search,  which  uses  pseudo  backbone 
frequencies  to  modify  the  way  that  WalkSAT  chooses  variables  to  flip.  The  second  phase  also  runs  many 
tries,  each  of  which  produces  a  local  minimum,  very  often  a  new  one.  The  newly  discovered  local  minima 
can  be  subsequently  added  to  the  pool  of  all  local  minima  found  so  far  and  be  used  to  update  the  pseudo 
backbone  frequencies. 

We  now  consider  methods  for  making  biased  moves  in  WalkSAT.  The  first  random  choice  in  WalkSAT 
is  clause  pick,  where  an  unsatisfied  clause  is  selected  when  multiple  ones  exist.  We  want  to  pick,  with 
high  probabilities,  those  variables  that  are  either  part  of  the  backbone  or  highly  constrained  in  all  optimal 
solutions.  Therefore,  we  should  choose  a  clause  containing  the  maximal  number  of  critically  constrained 
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variables.  To  this  end,  we  use  the  total  pseudo  backbone  frequency  of  all  the  literals  in  an  unsatisfied  clause, 
normalized  among  all  unsatisfied  clauses,  fo  measure  fhe  likelihood  fhaf  fhe  clause  confains  backbone  and 
highly  consfrained  variables.  We  fhen  selecf  an  unsafisfied  clause  among  all  unsalisfied  based  on  fheir 
likelihoods  of  confaining  backbone  variables.  Specifically,  lef  C  be  fhe  sef  of  unsafisfied  clauses,  and  qc  be 
fhe  sum  of  pseudo  backbone  frequencies  of  all  fhe  liferals  in  a  clause  C  ^  C.  We  fhen  lef  Pc  =  Qc/Q^^  the 
probabilify  fo  selecf  clause  C  among  all  unsafisfied  clauses  in  C,  where  Q  =  is  a  normalization 

factor. 

WalkSAT  uses  fhree  ofher  random  pick  rules  fo  arbifrarily  selecf  a  variable  affer  an  unsafisfied  clause 
is  chosen  (see  Section  5.1.2  and  Figure  45).  To  reiferafe,  fhe  fiaf  pick  rule  chooses  a  variable  from  a  sef 
of  zero  break-counf  variables,  if  any;  fhe  noise  pick  rule  selecfs  one  from  all  variables  involved  in  fhe 
chosen  clause;  and  fhe  greedy  pick  rule  fakes  a  variable  among  fhe  ones  of  leasf  break-counf.  In  essence, 
fhese  rules  use  fhe  same  operation,  picking  a  variable  equally  likely  from  a  sef  of  variables.  Therefore,  we 
can  modify  fhese  rules  all  in  fhe  same  way  by  using  pseudo  backbone  frequencies.  Lef  {xi,  X2,  •  •  •  ,Xw} 
be  a  sef  of  w  variables  from  which  one  musf  be  chosen,  {vi,V2,  -  ■  ■  ,  fheir  currenf  assignmenfs,  and 
{PIjP2^  •  •  •  jPw}  the  pseudo  backbone  frequencies  of  liferals  {(xi  =  xi),  (x2  =  X2),  •  •  •  ,  (x^  =  x^^,)}. 
Then  we  choose  variable  x*  wifh  probabilify  (1  —  Pi)/Y^^=i{^  ~  Pj)-  Here  we  use  probabilify  1  —  pi 
because  if  is  fhe  probabilify  of  liferal  (xj  =  ->Vi)  being  in  fhe  pseudo  backbone,  which  is  fhe  value  x*  is 
going  fo  change  fo. 

Furfhermore,  fhe  idea  of  pseudo  backbone  frequencies  can  also  be  applied  fo  generafe  an  inifial  assign- 
menf  for  a  local  search.  Specifically,  a  variable  is  assigned  a  particular  value  wifh  a  probabilify  proportional 
fo  fhe  pseudo  backbone  frequency  of  fhe  variable-value  pair.  This  was  called  heuristic  backbone  sampling 
in  [95]. 

5.3.4  Backbone  guided  WalkSAT  with  dynamic  noise 

To  make  fhe  backbone  guided  WalkSAT  algorifhm  more  general  and  robusf,  we  would  like  fo  have  if  use 
dynamic  noise  sfrafegy.  The  dynamic  noise  sfrafegy  discussed  in  Section  5.1.3  runs  a  long  sequence  of 
variable  selections  and  flips  wifh  no  resfarfs.  This,  unforfunafely,  is  incompatible  wifh  backbone  guided 
local  search,  which  requires  random  resfarfs  in  order  fo  collecf  local  minima  fo  consfrucf  pseudo  backbone 
frequencies. 

To  overcome  fhis  problem,  we  have  devised  a  “compromise”,  which  allows  a  reasonable  combinafion 
of  using  dynamic  noise  mefhod  and  applying  backbone  information.  Specifically,  we  run  WalkSAT  wifh 
dynamic  noise  sfrafegy  for  a  number  of  shorf  runs  fo  consfrucf  pseudo  backbone  frequencies,  followed  by 
several  long  runs  of  backbone  guided  local  search.  In  our  parficular  implemenfafion  of  backbone  guided 
Dyna- WalkSAT,  we  lef  if  run  fhirfy  shorf  runs  for  compufing  pseudo  backbone  frequencies,  followed  by 
seven  long  runs,  each  of  which  has  fen  fimes  more  flips  fhan  a  shorf  run. 
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5.3.5  Computing  pseudo  backbone  frequencies 

The  performance  of  backbone  guided  local  search  depends  greatly  upon  the  quality  of  pseudo  backbone 
information  used.  The  more  truthful  the  pseudo  backbone  information  is,  the  more  effective  the  new  local 
search  will  be.  In  order  to  retrieve  as  much  backbone  information  as  possible,  an  unbiased  sample  of  local 
minima  should  be  used,  in  which  local  minima  need  to  be  derived  from  independently  generated  starting 
assignments.  Therefore,  random  initial  assignments  should  be  preferred. 

Given  a  set  of  local  minima,  the  computation  of  pseudo  backbone  frequencies  needs  to  be  done  with 
care.  We  propose  two  different  ways  to  compute  pseudo  backbone  frequencies.  The  first  and  most  straight¬ 
forward  method  is  to  treat  all  the  given  local  minima  as  if  they  were  of  equal  quality,  and  take  the  frequency 
of  a  literal  I  that  appears  in  all  local  minima  S  as  its  pseudo  backbone  frequency  p{l).  Specifically,  we 
have 

p(/)  =  ^  (10) 

We  call  this  method  averaging  counting  or  AC  for  short. 

It  is  imperative  to  note  that  not  all  local  minima  are  of  equal  quality.  In  general,  a  lower  quality  local 
minimum  tends  to  contain  less  backbone  information  than  a  higher  quality  local  minimum,  as  discussed  in 
Section  5.2.  Therefore,  the  backbone  information  carried  in  a  lower  quality  local  minimum  is  less  reliable. 
This  means  that  a  literal  appearing  in  a  lower  quality  local  minimum  should  contribute  less  to  the  pseudo 
backbone  frequencies  than  a  literal  appearing  in  a  higher  quality  local  minimum.  As  a  result,  we  introduce 
a  discount  factor  to  adjust  the  contribution  of  a  literal  based  on  the  quality  of  the  local  minimum  where 
it  came  from.  If  a  local  minimum  Si  has  cost  q,  which  is  the  number  of  violated  clauses  in  the  local 
minimum,  then  we  can  compute  the  pseudo  backbone  frequency  p{l)  of  a  literal  I  =  {xi  =  Vi)  as  follows. 


In  other  words,  the  contribution  of  a  local  minimum  toward  a  pseudo  backbone  probability  of  a  literal  is 
reciprocally  weighted  by  the  cost  of  the  local  minimum.  We  thus  call  this  method  cost  reciprocal  averaging 
counting  (CRAG). 

5.4  Experimental  Evaluation 

We  now  experimentally  analyze  the  performance  of  the  backbone  guided  WalkSAT  (BG-WalkSAT)  algo¬ 
rithm  on  SAT  and  Max-SAT.  We  used  the  dynamic  noise  strategy  in  both  WalkSAT  and  BG-WalkSAT,  i.e., 
we  compare  Dyna- WalkSAT  and  BG-Dyna-WalkSAT.  Our  benchmark  problems  are  randomly  generated 
problems  and  those  from  the  SATLIB  [47]. 

5.4.1  Random  ensembles 

In  this  set  of  experiments,  we  generated  random  MAX-3-SAT  instances  with  2,000  variables  and  three 
different  CfV  ratios  of  4.3,  6.0,  and  8.0,  to  sample  instances  from  regions  of  differing  constrainedness.  We 
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Configuration 

C/V  ratio 

4.3 

6.0 

8.0 

Dyna-WalkSAT 

BG-Dyna-WalkSAT 

11.06  ±0.718 

23.28  ±  1.060 

200.87  ±4.411 

190.91  ±4.958 

531.79  ±  10.990 
504.21  ±  10.643 

BG-  GreedyPick 
BG-NoisePick 

BG -Initialization 
BG-ClausePick 

20.58  ±  1.046 
10.47  ±0.725 

7.33  ±  0.627 
13.75  ±  0.693 

200.01  ±4.499 
185.21  ±4.203 

190.290  ±4.19 
202.74  ±4.595 

506.68  ±  10.638 

518.48  ±10.827 

519.96  ±10.771 
537.02  ±11.156 

Table  4:  Comparison  of  backbone  guided  Dyna-WalkSAT  variations  over  Dyna-WalkSAT  on  2,000  vari¬ 
able  random  Max-3-SAT.  Performance  is  measured  by  the  average  number  of  constraint  violations.  The 
errors  represent  95%  confidence  intervals. 


ignored  C/V  ratio  of  2.0  since  WalkSAT  can  easily  find  safisfiable  assignmenfs  fo  mosf  of  such  undercon- 
sfrained  problems.  Af  each  of  fhe  rafios  considered,  we  generafed  1,000  problem  insfances,  and  compared 
Dyna-WalkSAT  and  BG-Dyna-WalkSAT.  Dyna-WalkSAT  ran  one  long  fry  wifh  a  maximum  of  one  mil¬ 
lion  flips.  BG-Dyna-WalkSAT  ran  30  shorf  fries,  each  wifh  a  maximum  of  10,000  flips,  as  in  WalkSAT, 
fo  collecf  local  minima.  If  fhen  execufed  seven  long  fries,  each  wifh  100,000  flips,  which  was  fen  limes 
longer  fhan  a  shorf  fry.  Thus  BG-Dyna- WalkSAT  executed  one  million  flips  also. 

We  found  fhaf  on  random  problem  insfances  fhe  average  counting  (AC)  melhod  for  computing  pseudo 
backbone  frequencies  is  less  effeclive  fhan  fhe  cosf  reciprocal  averaging  counfing  (CRAC)  mefhod  under 
all  differenl  C/V  ratios  we  fesfed.  Therefore,  we  will  presenf  fhe  resulfs  from  CRAC  here. 

Using  random  problem  insfances,  we  firsl  examined  fhe  effecls  of  applying  biased  moves  fo  Dyna- 
WalkSAT.  The  resulfs  are  included  in  Table  4.  In  fhe  fable,  we  lisf  fhe  resulfs  of  average  conslrainl  vio- 
lalions  for  Dyna-WalkSAT  and  BG-Dyna- WalkSAT  firsl,  followed  by  fhe  applications  of  biased  moves  fo 
differenl  random  picks  in  WalkSAT.  For  inslance,  BG-ClausePick  slands  for  Dyna-WalkSAT  wifh  biased 
clause  picks.  As  shown,  biased  noise  pick  and  biased  initialization  can  improve  Dyna-WalkSAT  under 
all  C/V  rafios;  biased  clause  pick  has  negalive  effecls  on  performance;  and  biased  greedy  pick  is  only 
effeclive  on  highly  overconslrained  problems.  Their  combinalion  also  has  mixed  effecls:  fhe  combined 
biased  moves  improve  upon  Dyna-WalkSAT  on  overconslrained  Max-3-SAT.  Nole  lhal  we  do  nof  include 
fhe  resulfs  for  biased  Hal  pick  because  if  has  no  effecl  on  almosl  all  problem  insfances  we  fesfed.  The 
reason  is  fhaf  in  mosf  cases,  Ihere  is  only  one  variable  wifh  zero  break-counl  for  3-SAT,  so  lhal  biased  flal 
pick  was  nof  used  mosf  of  fhe  lime. 

The  resulfs  in  Table  4  also  show  lhal  BG-Dyna-WalkSAT  can  only  improve  upon  Dyna-WalkSAT 
on  overconslrained  insfances  wifh  fhe  C/V  rafios  equal  fo  6.0  and  8.0;  while  if  fails  fo  do  so  on  critically 
consfrained  insfances  wifh  fhe  C/V  ralio  of  4.3.  One  possible  reason  is  lhal  Dyna-WalkSAT  is  very  effeclive 
and  efficienl,  finding  oplimal  solutions  very  often. 

Addilional  insighl  can  be  gained  from  an  inspection  of  anylime  behavior  of  Dyna-WalkSAT  and  BG- 
Dyna-WalkSAT,  which  are  shown  in  Figure  50.  As  discussed  earlier,  BG-Dyna- WalkSAT  oulperforms 
Dyna-WalkSAT  only  when  fhe  C/V  rafios  are  6.0  and  8.0.  A  key  observalion  on  Ihese  figures  is  lhal 
Ihere  was  a  big  jump  on  fhe  qualify  of  fhe  besl  local  minimum  found  so  far  righl  after  pseudo  backbone 


83 


250 


c/v=4.3 


c/v=6 


c/v=8 


Figure  50:  Anytime  performanee  of  Dyna-WalkSAT  and  BG-Dyna-WalkSAT  on  random  Max-3-SAT  with 
2,000  variables. 


information  was  applied  to  the  seareh  algorithm.  This  indieates  that  pseudo  baekbone  information  ean 
indeed  improve  the  seareh  performanee. 

We  also  investigated  the  performanee  of  BG-Dyna-WalkSAT  with  all  biased  moves  as  the  problem 
size  inereases.  We  eonsidered  random  Max-3-SAT  with  C/V  ratio  fixed  at  8.0,  and  the  number  of  variables 
from  2,000  to  10,000,  with  an  inerement  of  2,000.  We  used  1,000  problem  instanees  for  eaeh  different  size 
of  problems.  The  results  eomparing  to  Dyna-WalkSAT  are  shown  in  Table  5. 

As  the  results  showed,  BG-Dyna-WalkSAT  is  able  to  improve  upon  Dyna-WalkSAT  on  random  Max- 
3-SAT,  espeeially  on  overeonstrained  problem  instanees. 

5.4.2  Problem  instances  from  SATLIB 

We  eompared  BG-Dyna-WalkSAT  against  Dyna-WalkSAT  on  problem  instanees  from  SATLIB  [47].  The 
test  problems  inelude  SAT-eneoded  instanees  from  a  variety  of  applieation  domains,  ineluding  bloeks 
world  planning,  bounded  model  eheeking,  all  interval  series  problems,  and  hard  graph  eoloring  problems. 
We  only  ehose  problems  with  more  than  350  variables,  and  disearded  those  that  ean  be  easily  solved  by 
Walks  AT  and  BGWalkSAT.  The  ehosen  problem  instanees  are  diffieult  to  solve  in  general,  and  their  details 
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#  var 

Dyna-WalkSAT 

BG-Dyna-WalkSAT 

Dijf 

2,000 

528.32 

497.61 

30.71 

4,000 

1097.60 

1041.92 

55.68 

6,000 

1675.80 

1597.36 

78.44 

8,000 

2248.19 

2159.61 

88.58 

10,000 

2831.32 

2724.14 

107.18 

Table  5:  Comparison  of  BG-Dyna-WalkSAT  and  Dyna-WalkSAT  on  random  Max-3-SAT  with  C/V  ratio  of 
8.0,  averaged  over  1,000  instanees.  Diffis  the  improvement  of  BG-Dyna-WalkSAT  over  Dyna-WalkSAT. 


Table  6:  BG-Dyna-WalkSAT  versus  Dyna-WalkSAT  on  relatively  easy  satisfiable  problems.  Dyna- 
WalkSAT  and  BG-Dyna-WalkSAT  are  the  numbers  of  runs  resulting  in  satisfying  solutions  (out  of  20) 
by  these  algorithms.  The  better  results  from  the  two  algorithms  are  underlined  and  in  bold. 


problem 

#Var 

^Clause 

Dyna-WalkSAT 

BG-Dyna-WalkSAT 

bwJarge.c 

3016 

50457 

1 

2 

bw  large,  d 

6325 

131973 

1 

0 

par8-l 

350 

1149 

6 

19 

par8-2 

350 

1157 

6 

19 

par8-3 

350 

1171 

7 

12 

par8-4 

350 

1155 

0 

16 

par8-5 

350 

1171 

1 

15 

qgl-08 

512 

148957 

8 

12 

qg2-08 

512 

148957 

1 

4 

qg3-08 

512 

10469 

11 

20 

qg6-09 

729 

21844 

0 

5 

qg7-09 

729 

22060 

4 

5 

gl25.17 

2125 

66272 

5 

0 

g250.29 

7250 

454622 

4 

2 

ean  be  found  on  the  website. 

We  eonsidered  satisfiable  and  unsatisfiable  problems.  We  ran  both  Dyna-WalkSAT  and  BG-Dyna- 
WalkSAT  with  a  total  of  ten  million  flips  (eompared  with  one  million  for  our  results  for  random  instanees) 
as  most  of  these  problem  instanees  are  larger  than  the  random  Max-3-SAT  instanees  eonsidered  in  the 
previous  experiments.  Interestingly,  the  average  eounting  (AC)  method  for  eomputing  pseudo  baekbone 
frequeneies  is  slightly  better  than  the  eost  reeiproeal  averaging  eounting  (CRAC)  method.  Moreover, 
biased  noise  piek  and  biased  elause  piek  provide  substantial  improvements  to  Dyna-WaslkSAT;  their  eom- 
bination  exhibited  superior  performanee,  over  a  wide  range  of  real  instanees.  In  the  rest  of  the  seetion, 
we  present  the  results  of  BG-Dyna-WalkSAT  using  these  two  biased  moves.  In  our  experiments,  we  ran 
eaeh  of  BG- Dyna-WalkSAT  and  Dyna-WalkSAT  twenty  times,  with  eaeh  run  exeeuting  a  maximum  of  ten 
million  flips. 

In  viewing  the  results,  we  found  it  useful  to  divide  the  satishable  instanees  into  two  eategories,  the 
easier  instanees,  whieh  were  solved  at  least  onee  (Table  6),  and  the  harder  ones,  whieh  were  not  solved  by 
either  method,  in  any  of  their  runs  (Table  7).  Results  for  unsatishable  instanees  are  presented  in  Table  8. 

As  the  results  show,  BG-Dyna-WalkSAT  signiheantly  outperforms  Dyna-WalkSAT  in  most  eases. 
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Table  7:  Dyna-WalkSAT  vs.  BG-Dyna-WalkSAT  on  harder  satisfiable  problems.  Dyna-WalkSAT  and 
BG-Dyna- Walks  AT  are  the  average  numbers  of  violations  in  the  best  solutions  found  by  the  algorithms 
for  a  given  problem,  averaged  over  20  runs.  Gain  is  the  pereentage  improvement  of  BG-Dyna-WalkSAT 
over  Dyna-WalkSAT.  The  better  results  are  underlined  and  in  bold. 


problem 

#Var 

#Clause 

Dyna-WalkSAT 

BG-Dyna-WalkSAT 

gain  (%) 

bmc-ibm-1 

9685 

55870 

25.3 

4.15 

83.60 

bmc-ibm-2 

3628 

14468 

5.4 

12 

77.78 

bmc-ibm-3 

14930 

72106 

115.25 

19.7 

82.91 

bmc-ibm-4 

28161 

139716 

118.15 

38.9 

67.08 

bmc-ibm-5 

9396 

41207 

12.95 

1.25 

90.35 

bmc-ibm-6 

51654 

368367 

358.25 

103.6 

71.08 

bmc-ibm-7 

8710 

39774 

17.4 

63.22 

bmc-galileo-8 

58074 

294821 

65.65 

15.5 

76.39 

bmc-galileo-9 

63624 

326999 

95.95 

17.3 

81.97 

bmc-ibm-10 

61088 

334861 

406.15 

162.45 

60.00 

bmc-ibm-1 1 

32109 

150027 

439.8 

358.45 

18.50 

bmc-ibm-1 2 

39598 

19477 

554.65 

445.25 

19.72 

bmc-ibm-1 3 

13215 

6572 

88.05 

2^ 

96.93 

f2000 

2000 

8500 

2.2 

2.05 

6.82 

parl6-l-c 

317 

1264 

5.45 

5.35 

1.83 

parl6-l 

1015 

3310 

10.45 

9.45 

9.57 

parl6-2-c 

349 

1392 

6.2 

4.84 

par  16-2 

1015 

3374 

10.6 

10.4 

1.89 

parl6-3-c 

334 

1332 

6 

5.65 

5.83 

par  16-3 

1015 

3344 

10.45 

9.75 

6.70 

parl6-4-c 

324 

1292 

6.15 

10.57 

par  16-4 

1015 

3324 

10.4 

9.55 

8.17 

parl6-5-c 

341 

1360 

6.25 

6.05 

3.20 

par  16-5 

1015 

3358 

10.45 

9.85 

5.74 

par32-l-c 

1315 

5254 

21.7 

20.85 

3.92 

par32-l 

3176 

10277 

30.95 

30.25 

2.26 

par32-2-c 

1303 

5206 

21.15 

21.2 

-0.24 

par32-2 

3176 

10253 

32.1 

28.35 

11.68 

par32-3-c 

1325 

5294 

22.05 

21.3 

3.40 

par32-3 

3176 

10297 

32.95 

28.55 

13.35 

par32-4-c 

1333 

5326 

21.3 

21.4 

-0.47 

par32-4 

3176 

10313 

33.65 

29.4 

12.63 

par32-5-c 

1339 

5350 

23.15 

22.05 

4.75 

par 32- 5 

3176 

10325 

32.9 

30.3 

7.90 

Average 

29.82 
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Table  8:  Dyna-WalkSAT  vs.  BG-Dyna-WalkSAT  on  unsatisfiable  problems.  The  legend  is  the  same  as 
that  in  Table  7. _ 


problem 

#Var 

#Clame 

Dyna-WalkSAT 

BG-Dyna-WalkSAT 

gain  (%) 

longmult06 

2848 

8853 

lA 

1.65 

-10.00 

longmultO? 

3319 

10335 

2.05 

2.2 

-7.32 

longmultOS 

3810 

11877 

3.65 

2.65 

27.40 

longmult09 

4321 

13479 

6.75 

57.04 

longmultlO 

4852 

15141 

10.25 

45.37 

longmultl  1 

5403 

16863 

15.05 

9^ 

38.87 

longmultll 

5974 

18645 

17.8 

16.2 

8.99 

longmultl  3 

6565 

20487 

23.25 

21.4 

7.96 

longmultM 

7176 

22389 

32.6 

24.6 

24.54 

longmultl  5 

7807 

24351 

41.5 

30 

27.71 

ssa6288-047 

10410 

34238 

100.25 

89.7 

10.52 

Average 

20.01 

On  easier  satisfiable  instanees  (Table  6),  BG-Dyna-WalkSAT  finds  more  satisfying  solutions  than  Dyna- 
WalkSAT  for  all  parity  (par)  and  quasigroup  (qg)  elasses,  and  produees  similar  results  to  Dyna-WalkSAT 
on  bloeksworld  (bg)  instanees.  On  harder  satisfiable  instanees  (Table  7),  BG-Dyna-WalkSAT  outperforms 
Dyna-WalkSAT  in  all  but  two  of  34  instanees,  where  it  is  less  than  half  a  pereent  worse.  In  eontrast,  the 
overall  average  gain  is  about  30%,  and  the  gain  is  over  50%  in  1 1  of  them.  On  unsatisfiable  insfanees  (Ta¬ 
ble  8),  BG-Dyna- Walks  AT  produees  impressive  gains  on  longmulf  insfanees,  and  on  ssa6288-047,  wifh 
an  overall  average  gain  of  20%.  On  unsalisfiable  quasigroup  insfanees  (nof  shown),  BG-Dyna-WalkSAT’s 
performanee  was  similar  fo  fhaf  of  Dyna-WalkSAT.  The  performanee  of  BG-Dyna- WalkS  AT  is  never  more 
fhan  10%  worse  fhan  Dyna-WalkSAT  on  any  of  fhe  unsalisfiable  insfanees  we  sludied. 

The  mosl  glaring  failure  of  BG- Dyna-WalkSAT  is  on  fhe  salisfiable  insfanees  gl25.17  and  p250.29, 
shown  in  Table  6.  These  insfanees  are  SAT-eneoded  graph  eoloring  problems,  and  serve  fo  illuslrafe  an 
imporfanf  poinf.  As  deseribed  in  Seelion  5.2,  we  believe  fhaf  our  mefhod  is  effeelive  beeause  if  exploils 
fhe  “big  valley”  sfruefure  of  fhe  solufion  spaee.  However,  graph  eoloring  problems  exhibif  a  parfieular 
fype  of  symmefry  in  Iheir  solufion  slruelures  whieh  is  opaque  fo  loeal  seareh  melhods  sueh  as  WalkS  AT. 
For  example,  given  a  solufion  fo  a  graph  eoloring  problem,  swapping  red  wifh  green  resulfs  in  anofher 
solufion,  whieh  is  symmelrieal  fo  fhe  original.  Thus,  fhere  is  nof  a  single  “big  valley”  buf  several  in  fhe 
eonfiguralion  landseape  of  fhe  problem,  whieh  ean  bury  fhe  frue  baekbone  informalion  and  fhus  lead  fo 
degraded  performanee.  Presumably,  BG-Dyna-WalkSAT’s  performanee  will  suffer  on  all  insfanees  wifh 
fhis  fype  of  symmefry. 

5.5  Conclusions  and  Discussions 

In  Ibis  seelion,  we  firsl  earried  oul  a  syslemalie  invesligalion  of  eonfiguralion  landseapes  of  loeal  minima 
reaehed  by  fhe  WalkSAT  loeal  seareh  algorilhm  on  random  3-SAT  and  Max-3-SAT  problems.  In  Ibis 
analysis,  we  inlrodueed  eonfiguralion  landseapes  fo  eaplure  fhe  dislribulions  of  loeal  minima  in  lerms  of 
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their  cost  and  structural  differences.  Our  analysis  revealed  that  the  configuration  landscape  of  a  set  of  local 
minima  from  WalkSAT  exhibit  a  single  bell-shaped  surface,  showing  that  the  local  minima  form  a  single 
large  cluster.  Our  results  also  showed  that  the  WalkSAT  algorithm  is  effective  on  Max-3-SAT,  finding  high 
quality  local  minima  that  have  large  portions  of  variable  assignments  consistent  with  optimal  solutions. 

Based  on  the  configuration  landscape  analysis,  we  developed  a  novel  method  to  exploit  backbone  in¬ 
formation  to  improve  the  performance  of  a  local  search  algorithm,  the  WalkSAT  algorithm  in  particular. 
The  main  ideas  of  the  method  are  to  extract  backbone  information  from  local  minima  and  use  it  directly 
to  fix  possible  discrepancies  befween  fhe  currenf  assignmenf  and  opfimal  solufions,  so  as  fo  guide  a  local 
search  algorifhm  fowards  fhe  regions  of  search  space  confaining  high  qualify  as  well  as  opfimal  solu- 
fions.  Our  experimenfal  resulfs  show  fhaf  fhe  new  mefhod  can  significanlly  improve  fhe  performance  of 
fhe  WalkSAT  local  search  algorifhm  on  mosf  problem  insfances  from  SATLIB,  including  SAT-encoded 
problem  insfances  from  various  applicafions.  On  fhese  problem  insfances,  our  backbone  guided  WalkSAT 
algorifhm  has  a  higher  probabilify  of  reaching  safisfiable  solufions  fhan  fhe  original  WalkSAT  algorifhm, 
and  is  able  fo  improve  ifs  solution  qualify  on  Max-SAT  problem  insfances  by  20%. 

In  refrospecf,  fhe  mosf  imporfanf  confribufions  of  and  lessons  learnf  from  fhis  secfion  are  fhe  idea  of 
using  backbone  information  fo  improve  fhe  performance  of  a  local  search  algorifhm  and  a  simple  way  of 
capfuring  backbone  information  by  using  local  minima  from  a  local  search  algorifhm.  These  ideas  are 
general  and  applicable  fo  ofher  combinaforial  problems  and  ofher  search  mefhods.  For  example,  we  have 
successfully  applied  fhe  ideas  of  backbone  guided  local  search  fo  fhe  Traveling  Salesman  Problem  and 
fhe  Lin-Kernighan  local  search  algorifhm  [65] ,  which  is  one  of  fhe  oldesf  and  mosf  efficienf  algorifhm  for 
fhe  problem  [114].  By  using  sfrucfural  informalion  such  as  backbones,  fhe  new  mefhod  drives  a  search 
algorifhm  fowards  fhe  areas  of  fhe  search  space  where  mosf  opfimal  or  near  opfimal  solufions  are  locafed. 
In  comparison,  mosf  exisfing  search  fechniques  focus  on  fhe  cosfs  of  fhe  slates  in  a  search  space.  Therefore, 
fhe  new  algorifhm  is  focused  more  on  where  fhe  problems  are  in  fhe  currenf  slate,  and  fries  fo  fix  Ihem 
direclly. 

One  possible  drawback  of  our  mefhod  is  fhaf  if  requires  a  good  eslimalion  of  backbone  informalion. 
If  Ibis  estimation  deviates  subslanlially  from  fhe  real  backbone  information,  fhe  new  mefhod  will  nol  be 
effeclive.  Neverlheless,  fhe  cosl  reciprocal  mefhod  for  eslimale  backbone  frequencies  provides  a  simple 
mechanism  fo  ease  fhis  problem  fo  some  exlenl  by  discounling  fhe  confribulion  of  a  poor  local  minimum 
fo  fhe  pseudo  backbone  frequencies.  Furlhermore,  mosf  local  search  mefhods  are  randomized  algorilhms, 
so  belter  solufions  may  occasionally  be  discovered  and  added  fo  fhe  pool  of  local  minima.  Such  heller 
local  minima  will  subsequenlly  improve  fhe  qualify  of  fhe  estimation  of  backbone  information. 

The  new  backbone  guided  local  search  mefhod  seems  fo  be  nol  very  effeclive  on  Max-SAT  problems 
wilh  lillle  slruclure.  The  mefhod  is  particularly  hindered  by  symmelry  embedded  in  a  problem.  An  exam¬ 
ple  of  such  a  problem  is  graph  coloring,  where  swapping  Iwo  colors  in  a  solulion  leads  fo  anolher  solulion. 
In  shorl,  backbone  guided  local  search  seems  fo  be  effective  on  problems  from  which  significanl  sfrucfural 
informalion  can  be  exlracled.  How  fo  extend  fhe  ideas  and  algorifhm  presenled  in  fhis  secfion  fo  address 
symmefries  is  an  inleresling  fufure  research  topic. 


6  An  Improved  Integer  Local  Search  for  Complex  Scheduling  Problems 


The  recent  advances  in  the  research  of  Boolean  satisfiahility  (SAT)  have  provided  great  insights  into  the 
prohlem,  such  as  phase  transitions  and  hackhones  [77,  78,  109]  (also  discussed  in  Section  4),  and  have 
developed  efficient  algorithms  for  solving  SAT,  represented  hy  the  widely  applied  WalkSAT  local  search 
algorithm  [71,  87]  and  its  variants  [45,  71].  The  success  of  WalkSAT  has  also  led  to  the  paradigm  of 
formulating  and  solving  complex  planning  and  scheduling  problems  as  SAT  problems  [57,  59,  58].  Under 
this  paradigm,  a  complex  problem  is  encoded  as  a  SAT  problem,  a  solution  to  the  SAT  problem  is  found  by 
applying  an  algorithm  for  SAT,  and  finally  fhe  solufion  is  mapped  back  fo  fhe  original  problem.  This  SAT- 
based  paradigm  has  been  shown  successful  for  some  complex  problems  in  real  applicafions.  For  example. 
Blackbox  is  one  of  fhe  mosf  compefifive  mefhods  for  planning  [58,  59],  which  was  developed  under  fhe 
SAT-based  paradigm  by  applying  SAT  encoding  and  SAT  algorifhms. 

Many  consfrainfs  in  real-world  applicafions,  however,  are  complex  and  may  nol  be  easily  and  effi- 
cienfly  encoded  as  clauses  [15,  28].  More  useful  and  general  consfrainf  formulations  are  infeger  linear 
programs  (ILP)  [41,  98],  which  allow  infeger  variables  and  complex  consfrainfs,  and  subsume  pseudo 
Boolean  formulae  wifh  variables  faking  values  0  or  1  [39,  97,  98].  ILPs  and  pseudo  Boolean  formulae 
have  been  extensively  applied  fo  planning  and  scheduling  problems  [60,  98]. 

WSAT(oip)  [98]  is  an  extension  fo  fhe  WalkSAT  algorifhm  for  handling  overconsfrained  integer  programs 
(OIPs)  fhaf  involve  hard  and  sofl  consfrainfs.  Here  a  hard  consfrainf  is  one  fhaf  needs  fo  be  safisfied,  and 
a  soff  consfrainf  is  one  fhaf  may  be  violafed  buf  incurs  a  penalfy  if  nof  safisfied.  The  objecfive  of  such  a 
problem  is  fo  satisfy  all  hard  consfrainfs,  if  possible,  or  as  many  hard  consfrainfs  as  possible  when  over- 
consfrained,  while  minimizing  a  penally  funclion.  WSAT(oip)  has  been  shown  effeclive  on  large  consfrainf 
optimization  and  scheduling  problems  [60,  97,  98]. 

Inherited  from  fhe  WalkSAT  algorifhm,  WSAT(oip)  is  a  local  search  algorifhm  fhaf  makes  stochastic  local 
perfurbafions  fo  fhe  currenf  assignmenf  of  all  variables  in  searching  for  progressively  belter  solulions  [98]. 

A  noliceable  characlerislic  of  WalkSAT  and  WSAT(oip)  is  fhaf  whenever  multiple  choices  exisl,  a  uniformly 
random  or  unbiased  choice  will  be  made.  For  example,  when  an  unsalisfied  clause  is  to  be  selected  from 
a  sel  of  unsatisfied  clauses,  each  qualified  candidate  is  given  equal  change  fo  be  picked.  Likewise,  fhe 
variable  whose  value  is  to  be  changed  nexl  is  chosen,  uniformly  randomly,  from  mulliple  candidates.  Such 
uniform  random  moves  are  ineffeclive  when  fhere  exisl  large  “plaleau”  regions  in  search  space,  and  fhe 
problem  is  exacerbated  in  OIPs  and  WSAT(oip)  when  “plaleau”  regions  become  larger  due  fo  larger  domains 
of  infeger  variables. 

Molivaled  fo  solve  complex,  real-world  scheduling  problems  wifh  hard  and  sofl  consfrainfs,  we  aim 
fo  improve  WSAT(oip).  We  parlicularly  infroduce  Ihree  techniques  fo  fhe  existing  algorifhm.  The  firsl  is  a 
melhod  of  making  biased  moves  in  allempling  to  fix  possible  discrepancies  belween  fhe  currenf  variable  as- 
signmenl  and  an  oplimal  solufion,  so  as  fo  drive  fhe  search  to  fhe  regions  in  search  space  where  high-qualily 
and  oplimal  solulions  locate.  These  biased  moves  are  devised  based  on  our  previous  work  of  backbone 
guided  local  search  for  (maximum)  Boolean  salisfiabilily  [115].  The  second  melhod  is  a  sampling-based 
aspiralion  search  in  order  to  reslricl  fhe  search  fo  finding  progressively  improving  solutions,  so  as  fo  reduce 
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search  complexity  and  increase  anytime  performance  of  the  resulting  algorithm.  Our  experimental  anal¬ 
ysis  show  that  this  method  is  particularly  effective  on  problems  with  hard  and  soft  constraints.  The  third 
method  is  an  extension  of  Hoos’s  dynamic  noise  strategy  for  WalkSAT  [46]  to  WSAT(oip),  so  that  the  critical 
parameter  of  noise  ratio  of  WSAT(oip)  does  not  have  to  be  tuned  for  each  individual  problem  instance.  The 
resulting  WSAT(oip)  becomes  more  robust,  general  and  flexible  for  different  applications. 

This  section  is  organized  as  follows.  We  first  describe  in  Section  6.1  our  motivating  scheduling  prob¬ 
lem  and  consider  its  complexity.  We  then  discuss  pseudo  Boolean  encoding  and  overconstrained  integer 
programs  in  Section  6.2,  and  briefly  describe  WalkSAT  and  WSAT(oip)  in  Section  6.3.  We  then  present 
the  three  improving  techniques  for  WSAT(oip)  in  Section  6.4.  We  experimentally  evaluate  the  extended 
WSAT(oip)  in  Section  6.5,  using  our  scheduling  problems  and  the  instances  of  two  scheduling  problems 
from  CSPLIB  [36].  We  finally  conclude  in  Section  6.6. 

6.1  Scheduling  and  Resource  Allocation 

The  specific,  motivating  scheduling  problem  of  this  research  is  to  schedule  a  large  number  of  training 
activities  for  a  crew  over  a  period  of  time,  ranging  from  a  few  days  to  a  few  weeks  or  months  [15,  28]. 

In  such  a  problem,  a  trainee  needs  to  finish  a  set  of  required  activities  that  requires  many  trainers  and 
various  equipment.  These  activities  are  associated  with  one  another  by  precedent  relationships,  i.e.,  one 
training  activity  cannot  be  scheduled  until  a  trainee  has  finished  certain  prerequisites.  A  used  equipment 
(resources)  can  be  reused  after  some  maintenance,  which  itself  is  an  activity  to  be  scheduled.  In  addition, 
individual  activities  have  different  importance  and  carry  different  penalties  if  not  scheduled.  The  objective 
is  to  schedule  as  many  activities  as  possible  for  all  the  trainees  within  a  gross  period  of  time  using  the 
available  trainers  and  equipment  so  that  the  penalty  of  unscheduled  activities  is  minimized.  Even  though 
this  scheduling  problem  is  not  overarchingly  sophisticated,  it  can  indeed  be  viewed  as  a  representative  of 
general  scheduling  problems  with  various  constraints  and  being  required  to  optimize  an  objective  function. 

At  the  center  of  our  training  scheduling  problem,  as  well  as  many  other  similar  problems,  is  a  resource 
allocation  problem,  i.e.,  a  problem  of  assigning  resources  (e.g.,  trainers  and  equipment  in  our  scheduling 
problem)  to  needy  activities.  The  properties  of  such  an  underlying  resource  allocation  problem  can  help 
characterize  the  scheduling  problem.  The  complexity  of  the  former  will  dominate  the  complexity  of  the 
latter.  If  the  resource  allocation  problem  is  difficult,  the  scheduling  problem  is  doomed  to  be  hard  as  well. 

We  now  consider  a  simple,  static  resource  allocation  problem  that  was  abstracted  from  our  training 
scheduling  problem  at  a  particular  time.  We  are  given  a  set  of  n  tasks,  T  =  {Ti,  T2,  •  •  •  ,  T^},  and  a  set  of 
r  resources,  72.  =  { ,  • ' '  >  }  ■  Each  task  requires  a  certain  number  of  resources  in  order  to  execute , 

which  we  call  resource  requirements.  Each  resource  can  only  be  allocate  to  one  resource  requirement, 
and  a  resource  requirement  can  be  met  by  having  one  desirable  resource  allocated  to  it.  We  denote  the 
qi  resource  requirements  of  task  Tj  by  Q*  =  Qi,2,  •  •  •  ,  Qi,qi}-  Table  9  shows  a  small  example 

of  resource  requirements  of  two  tasks  over  three  resources.  An  entity  of  1  (0)  in  the  table  means  that  a 
resource  can  (cannot)  be  allocated  to  the  corresponding  requirement.  In  general,  the  available  resources 
may  not  be  sufficient  to  fulfill  every  task;  and  a  task  carries  a  penalty,  called  task  penalty,  if  not  scheduled. 
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Ri 

R2 

R3 

Ti 

Qi,i 

0 

1 

1 

Qi,2 

1 

0 

0 

T2 

Q2,1 

1 

1 

0 

Q2,2 

1 

1 

0 

Table  9:  A  simple  resouree  alloeation  problem. 

The  resouree  alloeation  problem  is  to  alloeate  the  resourees  to  the  tasks  so  that  the  overall  penalty  of 
unfulfilled  tasks  is  minimized,  whieh  eonstitutes  an  optimization  problem.  If  all  tasks  have  equal  penalties, 
it  is  equivalent  to  fulfill  the  maximal  number  of  tasks. 

Compared  to  some  other  resouree  alloeation  problems,  for  instanees  the  permutation  problems  eon- 
sidered  in  [91,  99],  our  problem  has  a  unique,  small  strueture  embedded  within  a  task.  A  task  ean  be 
seheduled  if  and  only  if  all  its  resouree  requirements  are  met.  We  eall  this  feature  bundled  resource  re¬ 
quirement.  Furthermore,  a  pair  of  resouree  requirements  have  an  exelusive  resouree  eontention  in  that  a 
resouree  aequired  by  one  requirement  eannot  be  alloeated  to  the  others.  We  eall  this  feature  exclusive  re¬ 
source  contention.  To  be  eonvenient,  we  eall  the  problem  bundled,  exclusive  resource  allocation  problem, 
or  BERAP  for  short. 

We  now  show  that  BERAP  is  NP-hard  [32].  To  this  end,  we  prove  that  a  deeision  version  of  the 
problem  is  NP-eomplete  [32].  A  simple,  speeial  deeision  version  of  BERAP  is  the  following.  Given  a  set 
of  tasks,  eaeh  of  whieh  has  a  set  of  resouree  requirements,  deeide  if  at  least  k  tasks  ean  be  fulfilled.  Here 
we  simply  eonsider  every  task  having  a  penalty  one  if  it  is  not  fulfilled. 

Theorem  6.1  BERAP  with  more  than  two  resource  requirements  per  task  is  NP-complete. 

Proof:  We  show  the  above  deeision  version  of  BERAP  is  NP-eomplete.  We  reduee  a  NP-eomplete  set 
paeking  problem  [32]  to  this  deeision  problem.  Given  a  eolleetion  S  of  finite  sets  and  a  positive  integer 
K  <  |5|,  set  paeking  is  to  deeide  if  S  eontains  at  least  K  mutually  disjoint  subsets.  Eormally,  it  is  to  deeide 
if  there  exists  S'  C  S  sueh  that  IS'I  >  K  and  for  all  Si  £  S'  and  S2  £  S',  SiCi  S2  =  0.  The  problem  is 
NP-eomplete  when  every  subset  Si  £  S  has  more  than  two  elements.  We  now  reduee  an  NP-eomplete  set 
paeking  problem  to  our  deeision  BERAP.  We  map  all  the  elements  in  the  subsets  of  a  set  paeking  problem 
instanee  to  the  resourees  of  BERAP,  eaeh  subset  of  the  set  paeking  instanee  to  a  task  of  BERAP,  and  an 
element  in  the  subset  to  a  resouree  requirement  of  the  respeetive  task.  In  other  words,  the  total  number  of 
tasks  is  the  number  of  subsets  |5|,  the  number  of  resourees  is  the  number  of  distinet  elements  in  all  subsets 
of  S,  and  the  number  of  resouree  requirements  of  a  task  is  the  number  of  elements  in  the  eorresponding 
subset.  Given  K  <  151,  the  eonstrueted  BERAP  is  to  deeide  if  at  least  K  tasks  ean  be  fulfilled.  Clearly,  a 
solution  to  the  BERAP  is  also  a  solution  to  the  original  set  paeking  problem.  □ 

This  NP-eompleteness  result  leads  to  the  eonelusion  that  our  erew  seheduling  problem  in  intraetable 
in  the  worst  ease. 
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6.2  PB  Encoding  and  Integer  Programs 

A  clause  of  Boolean  variables  can  be  formulated  as  a  linear  pseudo  Boolean  (PB)  constraint  [39,  98], 
which  we  illustrate  by  an  example.  We  start  by  viewing  Boolean  value  True  (T)  as  integer  1,  and  value 
False  (F)  as  0.  We  then  map  a  Boolean  variable  v  to  an  integer  variable  x  that  takes  value  1  or  0,  and  map 
u  to  1  —  X.  Therefore,  when  u  =  T,  we  have  x  =  1  and  1  —  X  =  0  which  corresponds  to  v  =  F.  With 
this  mapping,  we  can  formulate  a  clause  in  a  linear  inequality.  For  example,  (m  V  T2  V  V3)  can  be  mapped 
to  xi  +  (1  —  X2)  +  X3  >  1.  Here,  the  inequality  means  that  the  clause  must  be  satisfied  in  order  for  the 
left  side  of  the  inequality  to  have  a  value  no  less  than  one.  In  general,  the  class  of  linear  PB  constraints  is 
defined  as  Ci  ■  Li  ^  d,  where  Cj  and  d  are  rational  numbers,  ~  belongs  to  {=,  <,  <,  >,  >},  and  the  Lj 
are  literals. 

However,  a  clause  in  an  overconstrained  problem  may  not  be  satisfied  so  that  its  corresponding  in¬ 
equality  may  be  violated.  To  represent  this  possibility,  we  introduce  an  auxiliary  integer  variable  w  to 
the  left  side  of  a  mapped  inequality.  Variable  m  =  1  if  the  corresponding  clause  is  unsatisfied,  mak¬ 
ing  the  inequality  valid;  te  =  0  otherwise.  Since  the  objective  is  to  minimize  the  number  of  violated 
clauses,  it  is  then  to  minimize  the  number  of  auxiliary  variables  that  are  forced  to  take  value  1.  To 
be  concrete,  (ui  V  V2  V  U3),  (r;2  V  ^4)  can  be  written  as  an  overconstrained  PB  formula  of  minimizing 
W  =  Cl  ■  wi  +  C2  ■  W2,  subject  to 

f  xi  +(1  -  X2)  +X3  +r(;i  >  1 

\  X2  +(1  -  X4)  +W2  >  1 

where  Ci  and  C2  are  the  penalties  of  the  first  and  second  clauses,  respectively. 

More  complex  constraint  problems,  where  variables  takes  integers  rather  than  Boolean  values,  can  be 
formulated  as  overconstrained  integer  programs  (OIPs)  [98],  which  are  integer  linear  programs  (ILPs)  [41] 
in  the  sense  that  they  both  use  inequalities  to  define  the  feasible  regions  of  a  solution  space  and  aim  to 
optimize  an  objective  function.  OIPs  differ  from  ILPs  in  that  OIPs  introduce  additional,  competing  soft 
constraints  to  encode  the  overall  optimization  objective. 

A  constraint  in  OIP  defines  a  feasible  region  for  all  assignments  of  the  (integer)  variables  involved.  For 
an  assignment  that  violating  a  constraint,  we  can  define  the  distance  of  the  assignment  to  the  boundary  of 
the  feasible  region  specified  by  the  constraint.  Such  a  distance  can  be  measured  by  the  Manhattan  distance, 
the  minimum  integer  distance  in  the  grid  space  defined  by  the  variable  domains.  This  Manhattan  distance 
was  call  score  of  the  constraint  under  the  given  assignment  [98].  Obviously,  if  an  assignment  satisfies  a 
consfraint,  fhen  its  distance  to  the  constraint  boundary  is  zero. 

6.3  The  WalkSAT  and  WSAT(oip)  Algorithms 

WSAT(oip)  belongs  to  the  family  of  WalkSAT -based  local  search  algorithms,  each  of  which  follows  the  same 
basic  procedure  of  the  WalkSAT  algorithm  [71,  87].  The  WalkSAT  algorithm  was  discussed  in  Section  5.1.2 
and  in  Figure  45. 

WSAT(oip)  was  built  to  solve  OIPs  by  extending  the  WalkSAT  algorithm  to  support  integer  variables  and 
generic  constraints  such  as  inequalities.  The  main  extensions  and  modifications  made  by  WSAT(oip)  are  the 
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following: 

•  Distinguishing  hard  and  soft  constraints'.  When  Choosing  a  violated  eonstraint  C,  WSAT(oip)  seleets 
a  violated  hard  eonstraint  with  probability  ph  and  a  violated  soft  eonstraint  with  probability  1  —  ph- 

•  Restricted  neighborhood:  When  ehoosing  a  variable  whose  value  to  be  ehanged  from  all  the  vari¬ 
ables  assoeiated  with  the  seleeted  eonstraint  C,  only  will  the  (integer)  values  that  differs  from  the 
eurrent  value  by  at  most  d  be  eonsidered. 

•  Tabu  search:  When  multiple  variable-value  pairs  exist  in  the  greedy  ehoiee  whieh  make  the  same 
amount  of  improvement  to  the  objeetive  value,  break  ties  first  in  favor  of  the  one  that  has  been  used 
the  least  frequently,  and  then  in  favor  of  the  one  that  has  not  been  used  the  longest. 

Similar  to  WalkSAT  WSAT(oip)  still  uses  a  noise  ratio,  whieh  has  to  be  tuned  for  every  problem  instanee. 

6.4  Improvement  and  Extensions  to  WSAT(oip) 

WSAT(oip)  is  not  very  effieient  on  large  problems.  We  introduee  three  extensions  to  improve  its  perfor- 
manee. 

6.4.1  Backbone-guilded  biased  moves 

One  observation  on  loeal  seareh  for  SAT  problems  is  that  there  exist  a  large  amount  of  plateau  regions 
in  the  seareh  spaee  where  neighboring  states  all  have  the  same  quality.  This  observation  inspired  the 
development  of  WalkSAT  that  “walks”  on  the  plateau,  thus  the  name  of  WalkSAT,  by  making  random  moves 
in  order  to  navigate  through  plateau  regions  and  to  hopefully  find  downfall  edges.  Sueh  random,  sometimes 
aimless,  plateau  moves  are  nol  very  effeelive.  Even  fhough  fhe  use  of  a  fabu  lisf  ean  help  prevenf  fo  visif 
fhe  reeenfly  visifed  sfafes  [45],  fhe  algorifhm  may  sfill  have  fo  explore  a  large  porfion  of  a  plateau  area. 

The  ineffieaey  of  WalkSAT's  random  moves  is  exaeerbafed  in  WSAT(oip)  where  non-Boolean  variables 
ean  have  large  domains,  whieh  lead  fo  larger  neighborhoods  and  fhus  larger  plateau  regions.  Therefore,  if 
is  imporfanf  fo  shorfen  or  avoid,  if  possible,  sueh  random  moves. 

The  main  ideas 

Our  main  idea  fo  address  fhe  ineffieaey  eaused  by  uniformly  random  moves  in  WSAT(oip)  is  fo  exploif 
an  extended  eoneepf  of  baekbone.  The  baekbone  variables  of  an  opfimizafion  problem  are  fhe  ones  fhaf 
have  fixed  values  among  all  opfimal  solufions;  and  fhese  baekbone  variables  are  eolleefively  ealled  fhe 
baekbone  of  fhe  problem.  The  size  of  fhe  baekbone,  fhe  fraelion  of  baekbone  variables  among  all  variables, 
is  a  measure  of  fhe  eonsfrainedness  of  a  given  problem.  The  eoneepf  of  baekbone  variables  ean  be  exfended 
fo  baekbone  frequeneies.  The  baekbone  frequeney  of  a  variable- value  pair  is  fhe  frequeney  fhaf  fhe  pair 
appears  in  all  opfimal  solutions;  and  fhe  baekbone  frequeney  of  a  variable  is  fhe  maximum  baekbone 
frequeney  of  ifs  values.  Speeifieally,  lef  x  be  a  variable  wifh  domain  D  =  {m,  U2,  •  •  •  ,  Vk},  and  p{x{vi)) 
be  fhe  baekbone  frequeney  of  x  faking  Vi,  fhen  fhe  baekbone  frequeney  of  x  is  p{x)  =  Taaxy^^£){p{x{vi))} . 
Thus,  a  baekbone  variable  musf  have  baekbone  frequeney  of  one.  The  baekbone  frequeney  of  a  variable 
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captures  the  tightness  of  the  constraints  that  the  variable  is  involved;  the  higher  the  frequency,  the  more 
constrained  the  variable  is. 

We  can  apply  backbone  frequencies  to  modify  random  moves  in  WSAT(oip).  If,  somehow,  we  knew 
the  backbone  frequencies  of  the  variable- value  pairs  of  a  problem,  we  could  construct  a  “smart”  search 
algorithm  by  using  the  backbone  frequency  information  as  an  oracle  to  guide  each  step  of  WSAT(oip).  At 
each  step  of  the  algorithm,  we  can  use  the  backbone  frequencies  to  change  the  way  in  which  a  variable  is 
chosen  to  focus  on  fixing  the  critically  constrained  variables  that  are  not  currently  set  correctly. 

Unfortunately,  obtaining  the  exact  backbone  frequencies  of  a  problem  requires  to  find  all  optimal 
solutions,  thus  is  more  difficult  than  finding  jusf  one  solution.  To  address  fhis  problem,  fhe  second  key 
idea  of  backbone  guided  local  search  is  fo  esfimafe  backbone  frequencies  using  local  minima  from  a  local 
search  algorifhm.  We  simply  freaf  local  minima  as  if  fhey  were  opfimal  solufions  and  compufe  pseudo 
backbone  frequencies,  which  are  an  esfimafe  of  real  backbone  frequencies.  More  precisely,  we  define 
fhe  pseudo  backbone  frequency  of  a  variable-value  pair  as  fhe  frequency  fhaf  fhe  pair  appears  in  all  local 
minima. 

The  qualify  of  pseudo  backbone  frequencies  depends  on  fhe  effectiveness  and  efficiency  of  fhe  local 
search  algorifhm  used.  High-qualify  local  minima  can  be  obfained  by  efficienf  local  search  algorifhms. 
Even  fhough  WSAT(oip)  may  land  on  subopfimal  solufions  wifh  fairly  high  probabilifies,  mosf  of  fhe  local 
minima  from  WSAT(oip)  are  expecf  fo  have  large  portions  of  variables  sef  fo  correcf  values,  so  fhaf  fhey 
confain  parfial  opfimal  solufions  or  parfial  backbone.  In  fhis  research,  we  direcfly  adopf  WSAT(oip)  fo 
collecf  local  minima,  and  fhen  in  refurn  apply  fhe  backbone  guided  search  mefhod  fo  fhe  algorifhm  fo 
improve  ifs  performance. 

Biased  moves  in  WSAT(oip) 

Pseudo  backbone  frequencies  can  be  incorporafed  in  WSAT(oip)  fo  make  ’’biased”  moves.  Consider  an 
example  of  fwo  variables,  xi  and  X2,  thaf  appear  in  a  violafed  consfrainf  and  have  fhe  same  effecf  under  fhe 
currenf  assignmenf,  i.e.,  changing  fhe  value  fo  one  of  fhem  makes  fhe  violafed  consfrainf  safisfied,  and  bofh 
variables  have  fhe  same  break-counf  or  will  cause  fhe  same  number  of  safisfied  consfrainfs  unsatisfied  if 
changed.  Lef  B  be  fhe  sef  of  backbone  variables  along  wifh  fheir  fixed  values,  T  be  fhe  sef  of  local  minima 
from  which  pseudo  backbone  frequencies  were  compufed,  and  vi  and  V2  are  fhe  currenf  values  of  xi  and 
X2-  We  will  prefer  fo  change  xi  over  X2  if  under  fhe  currenf  assignmenf,  P{(xi  =  vi)^B\'T'\  <  P{{x2  = 
V2)^B\T},  which  means  fhaf  under  fhe  currenf  assignmenf,  xi  is  less  likely  fo  be  parf  of  backbone  fhan 
X2,  given  fhe  sef  of  local  minima  T.  Nofe  fhaf  P{{x  =  v)^B\T]  is  fhe  pseudo  backbone  frequency  of 
X  =  V  under  fhe  evidence  of  a  sef  of  local  minima  T. 

How  can  fhe  pseudo  backbone  frequencies  be  used  fo  alfer  fhe  way  fhaf  WSAT(oip)  chooses  variables? 
As  discussed  in  Section  6.3,  WSAT(oip)  uniformly  randomly  chooses  a  variable  when  multiple  choices  exisf. 
For  example,  when  fhere  are  multiple  variables  wifh  zero  break-counf,  WSAT(oip)  chooses  one  arbifrarily. 
In  backbone  guided  search,  we  apply  pseudo  backbone  informafion  fo  force  WSAT(oip)  fo  make  random 
buf  biased  choices.  If  fwo  variables  can  make  a  consfrainf  safisfied,  fhe  variable  having  a  higher  backbone 
frequency  will  be  chosen.  In  ofher  words,  we  modify  WSAT(oip)’s  random  sfrafegies  in  such  a  way  fhaf  a 
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backbone  or  critically  constrained  variable  will  be  chosen  more  often  than  a  less  restricted  variable.  To 
this  end,  we  use  pseudo  backbone  frequencies  to  help  make  random  biased  selections. 

Specifically,  we  apply  pseudo  backbone  frequencies  to  modify  the  random  choices  made  in  WSAT(oip). 
The  first  random  choice  in  WSAT(oip)  is  constraint  pick,  where  a  violated  constraint  is  selected  if  multiple 
ones  exist.  We  want  to  pick,  with  high  probabilities,  those  variables  that  are  part  of  the  backbone  or 
highly  constrained  in  all  optimal  solutions.  Therefore,  we  choose  a  constraint  with  the  maximal  number 
of  critically  constrained  variables.  We  use  the  pseudo  backbone  frequencies  of  variables  in  an  unsatisfied 
consfrainf,  normalized  among  fhe  violafed  consfrainfs  involved,  fo  measure  fhe  degree  of  consfrainedness 
of  fhe  consfrainf.  We  fhen  selecf  an  unsatisfied  consfrainf  among  all  violated  ones  based  on  fheir  degrees  of 
consfrainedness.  Specifically,  lef  C  be  fhe  sef  of  unsalisfied  consfrainfs,  and  qc  fhe  sum  of  pseudo  backbone 
frequencies  of  all  fhe  variables  in  a  consfrainf  C  G  C.  We  fhen  selecf  consfrainf  C,  wifh  probabilify 
Pc  =  Qc/Q,  from  all  unsalisfied  consfrainfs  in  C,  where  Q  =  Xlcec  'fc  is  a  normalizalion  faclor. 

WSAT(oip)  uses  Ihree  olher  random  rules  fo  arbifrarily  selecf  a  variable  afler  an  unsalisfied  consfrainf  is 
chosen  (see  Seclion  6.3  and  Figure  45).  The  fial  pick  rule  chooses  a  variable  from  a  sef  of  zero  break-counl 
variables,  if  any;  fhe  noise  pick  rule  selecls  one  from  all  variables  involved  in  fhe  chosen  consfrainf;  and 
fhe  greedy  pick  rule  lakes  a  variable  among  fhe  ones  of  leasl  break-counl.  In  essence,  Ihese  rules  use  fhe 
same  operation,  i.e.,  picking  a  variable  equally  likely  from  a  sef  of  variables.  Therefore,  we  can  modify 
fhese  rules  all  in  fhe  same  way  by  using  pseudo  backbone  frequencies.  Lef  {xi,X2,  ■  ■  ■  ,  be  a  sef  of 
w  variables  from  which  one  musl  be  chosen,  {vi,V2,  -  ■  ■  ,  fheir  besl  satisfying  assignmenls  (fhe  ones 
satisfying  fhe  consfrainf  and  having  fhe  highesl  pseudo  backbone  frequencies),  and  {pi,P2,  •  •  •  ,Pw}  the 
pseudo  backbone  frequencies  of  variable-value  pairs  {(xi  =  xi),  (x2  =  X2),  •  •  •  ,  (x^„  =  x^)}.  Then  we 
choose  Xi  wifh  probabilify  pi/  Pj- 

Furlhermore,  fhe  idea  of  pseudo  backbone  frequencies  can  also  be  applied  fo  generate  an  inilial  assign- 
menl  for  a  local  search.  Specifically,  a  variable  is  assigned  a  particular  value  wifh  a  probabilify  proportional 
fo  fhe  pseudo  backbone  frequency  of  fhe  variable-value  pair. 

The  backbone  guided  WSAT(oip)  algorifhm 

The  backbone  guided  WSAT(oip)  algorifhm  has  fwo  phases.  The  firsf  is  fhe  estimation  phase  fhaf  collecfs 
local  minima  by  running  WSAT(oip),  wifh  a  fixed  number  of  fries.  The  local  minima  fhus  collecfed  are 
compiled  fo  compute  fhe  pseudo  backbone  frequencies  of  all  variable-value  pairs. 

The  second  phase  carries  ouf  fhe  acfual  backbone  guided  search,  which  uses  pseudo  backbone  frequen¬ 
cies  fo  modify  fhe  way  fhaf  WSAT(oip)  chooses  variables  fo  change.  This  phase  also  runs  many  fries,  each 
of  which  produces  a  (new)  local  minimum.  The  newly  discovered  local  minima  are  subsequenfly  added  fo 
fhe  pool  of  all  local  minima  found  so  far  fo  update  fhe  pseudo  backbone  frequencies. 

6.4.2  Aspiration  search 

Solving  an  OIP  requires  fo  opfimize  fwo  (conflicfing)  objecfives,  salisfying  fhe  maximum  number  of  hard 
consfrainfs  and  minimizing  a  penalfy  funcfion  of  soff  consfrainfs  violafed.  Two  obvious  mefhods  can 
be  faken  fo  make  a  balance  befween  fhese  fwo  objecfives.  One  is  fo  direcfly  search  for  a  solufion  by 
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considering  hard  and  soft  constraint  together,  which  was  suggested  and  taken  in  WSAT(oip)  [98],  which 
attempts  to  select  a  variable  involved  in  a  hard  constraint  with  probability  ph  and  a  variable  associated 
with  a  soft  constraint  with  probability  1  —  ph-  Note  that  the  performance  of  WSAT(oip)  depends  to  a  large 
degree  on  this  parameter.  The  other  way  is  to  satisfy  the  maximum  number  of  hard  constraints  first  and 
then  try  to  minimize  the  total  penalty  of  violated  soft  constraints.  However,  these  two  methods  do  not 
work  very  well  on  large,  complex  OIPs. 

In  many  real-world  constraint  problems,  our  training  scheduling  problems  discussed  in  Section  6.1 
in  particular,  the  number  of  hard  constraints  may  be  large;  finding  the  best  assignment  to  the  variables 
involved  in  hard  constraints  may  itself  be  a  costly  task.  Even  if  such  an  optimal  solution  can  be  found,  it 
may  not  be  extended  to  an  overall  assignment  of  minimal  penalty.  Therefore,  many  optimal  assignments 
to  the  variables  in  hard  constraints  must  be  examined,  making  the  overall  search  prohibitively  costly. 

To  make  WSAT(oip)  efficient  on  OIPs,  we  propose  what  we  call  aspiration  search  strategy.  An  aspi¬ 
ration  level  corresponds  to  a  targeted  penalty  score;  the  higher  an  aspiration  level,  the  lower  the  targeted 
penalty  score.  Given  an  aspiration  level,  we  first  search  for  an  assignment  so  that  the  total  penalty  of 
unsatisfied  soft  constraints  is  no  more  than  the  targeted  penalty  value.  When  such  an  assignment  is  found, 
we  attempt  to  extend  the  current  partial  assignment  to  satisfy  the  maximum  number  of  hard  constraints. 
This  process  of  extending  a  partial  assignment  to  a  complete  assignment  may  be  repeated  many  times; 
and  the  maximum  number  of  hard  constraints  satisfied  Sh  is  recorded.  Each  such  process  corresponds  fo 
a  probing  in  the  search  space  under  the  current  aspiration  level.  We  then  increase  the  aspiration  level  and 
repeat  the  processes  of  probing  with  the  objective  of  finding  an  assignment  that  violates  no  more  less  than 
Sh  hard  constraints  and  whose  penalty  meets  the  restriction  of  the  current  new  aspiration  level.  Sh  is  also 
updated  if  a  better  assignment,  one  violating  less  hard  constraints,  is  found  under  the  current  aspiration 
level.  This  means  that  the  overall  processes  attempt  to  find  progressively  better  solutions  for  both  hard  and 
soft  constraints.  If  we  fail  to  find  an  assignment  satisfying  at  least  Sh  hard  constraints  and  keeping  the 
penalty  above  the  current  aspiration  level  after  a  certain  number  of  tries,  the  algorithm  terminates  and  the 
best  solution  found  so  far  is  returned. 

Aspiration  search  has  several  advantages.  Eirst,  it  decomposes  an  OIP  into  several  decision  problems, 
each  of  which  has  a  different  degree  of  constrainedness  represented  by  an  aspiration  level.  At  a  given 
aspiration  level,  this  strategy  also  integrates  a  sampling  method,  which  first  probes  the  search  space  to 
reach  a  partial  assignment  such  that  the  penalty  function  is  above  the  aspiration  level,  with  a  search  for  sat¬ 
isfaction  of  the  hard  constraints.  Thanks  to  the  partial  assignment,  the  hard  constraints  can  be  simplified, 
as  fhe  variables  involved  in  soft  constraints  are  instantiated,  so  that  optimizing  hard  constraints  becomes 
relatively  easier.  As  a  result,  aspiration  search  is  able  to  reduce  search  cost.  Second,  the  probability  ph  of 
choosing  a  variable  involved  in  hard  constraints,  an  important  parameter  determining  the  performance  of 
WSAT(oip),  disappears,  making  the  algorithm  less  problem  dependent.  Third,  the  aspiration  search  strategy 
can  interact  closely  with  the  backbone-guided  local  search  method,  making  the  latter  more  effective.  Since 
aspiration  search  is  able  to  reach  progressively  better  solutions,  the  suboptimal  solutions  at  various  aspi¬ 
ration  levels  can  thus  be  used  as  local  minima  to  compute  pseudo  backbone  frequencies,  which  expedites 


96 


the  process  of  gathering  backbone  frequency  information. 

When  applying  sampling  method,  it  is  desirable,  albeit  difficult,  to  know  if  the  current  partial  assign¬ 
ment  at  a  certain  aspiration  level  can  be  extended  to  satisfy  at  least  Sh  hard  constraints.  Our  approach 
to  this  problem  is  to  monitor  the  progress  of  extending  the  partial  assignment  to  a  full  assignment.  If  the 
number  of  violated  hard  constraints  decreases  after  a  fixed  number  of  moves  M,  we  consider  fhe  currenf 
partial  assignmenf  extensible.  Ofherwise,  fhe  currenf  partial  assignmenf  will  be  abandoned,  and  anofher 
partial  assignmenf  above  fhe  currenf  aspirafion  level  will  be  sampled.  Note  fhaf  fhe  performance  of  fhe 
overall  search  is  affecled  by  fhe  fixed  number  of  moves  M  wifhin  which  a  beffer  complefe  assignmenf 
musf  be  found.  If  fhis  number  is  loo  large,  we  may  wasfe  too  much  lime  on  an  unsalisfiable  deadend; 
whereas  if  if  is  loo  small,  we  may  miss  a  salisfiable  sample.  We  develop  a  dynamic  melhod  to  adjusl  fhis 
parameler  M  in  our  exlended  WSAT(oip)  algorilhm.  The  delail  of  fhis  melhod  will  be  discussed  in  fhe  nexl 
subsection  where  we  collectively  deal  wilh  fhe  issues  of  how  lo  dynamically  adjusl  fhe  paramelers  of  fhe 
WSAT(oip)  and  extended  WSAT(oip)  algorilhm. 

6.4.3  Dynamic,  adaptive  parameters 

One  limitation  of  the  WalkSAT  family  of  algorithms,  including  WSAT(oip),  is  its  dependence  on  a  manually 
set  noise  ratio,  which  is  the  probability  of  how  often  a  nongreedy  move  should  be  taken  (see  Section  6.3). 

In  addition,  whenever  a  noise  ratio  is  chosen  it  will  be  used  throughout  the  search.  It  is  evident  that  big 
progresses  can  be  more  easily  made  at  an  early  stage  of  a  local  search  than  at  a  late  stage.  Therefore  the 
noise  ratio  should  be  adjusted  dynamically  depending  on  where  the  current  search  is  in  the  overall  search 
space. 

The  dynamic  noise  strategy  proposed  in  [46]  for  WalkSAT  is  one  such  method.  The  idea  of  this  strategy 
is  simple:  start  a  local  search  with  the  noise  ratio  equal  to  zero,  and  examine  the  number  of  violations 
in  the  current  state  every  9m  flips,  where  m  is  the  number  of  constraints  of  a  given  problem,  and  6  a 
constant.  If  the  number  of  violations  has  not  decreased  since  the  last  time  we  checked  {9m  flips  ago),  the 
search  is  assumed  to  have  stagnated,  and  the  noise  ratio  is  increased  to  wp  +  (1  —  wp)(j),  where  wp  is  the 
current  noise  ratio  and  (/>  is  another  constant.  Otherwise,  the  noise  ratio  is  decreased  to  wp{l  —  2(/)).  The 
discrepancy  between  the  formulas  for  increasing  and  decreasing  the  noise  ratio  is  based  on  some  empirical 
observations  of  how  WalkSAT  behaves  when  the  noise  ratio  is  too  high,  compared  with  how  it  behaves  when 
the  parameter  is  too  low  [46].  We  refer  to  this  strategy  as  Dyna-WalkSAT  for  convenience. 

Dyna-WalkSAT  uses  two  parameters,  9  and  cj).  The  difference  of  using  these  two  new  parameters  and 
using  the  noise  ratio  in  the  original  algorithm  is  that  these  two  parameters  do  not  have  to  be  tuned  for  every 
single  problem  instance;  the  performance  of  Dyna-WalkSAT  with  the  same  values  for  9and  <p  is  relatively 
consistent  across  different  problem  instances. 

Although  Dyna-WalkSAT  was  originally  designed  and  tested  on  WalkSAT  for  SAT,  we  have  found  it 
effective  on  WSAT(oip)  for  pseudo  Boolean  encoded  problems  and  OIPs.  We  call  WSAT(oip)  using  the 
dynamic  noise  strategy  Dyna-WSAT(oip).  Following  [46]  we  set  9=  1/6  and  i/)  =  1/5  in  Dyna-WSAT(oip), 
which  have  been  found  to  be  effective  over  a  wide  range  of  problem  instances.  Due  to  its  simplicity  and 
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Problem 

WSAT(oip) 

EWSAT(oip) 

n 

m 

unsat 

penalty 

time 

unsat 

penalty 

time 

11520 

16308 

12.43 

10380 

287.4 

12.08 

9330 

167.6 

21240 

34908 

10.58 

8550 

824.7 

8.05 

6270 

490.2 

21800 

37501 

3.45 

2520 

56.0 

3.35 

2325 

56.3 

40718 

72071 

3.95 

2970 

26.2 

3.88 

2880 

41.2 

41496 

66892 

11.93 

10230 

1936.4 

8.70 

6810 

965.2 

79580 

143896 

4.10 

3120 

514.5 

3.97 

2963 

344.3 

Table  10:  Comparison  on  crew  training  scheduling,  where  n  and  m  are  the  numbers  of  variables  and 
clauses,  respectively;  unsat  is  the  average  number  of  violated  hard  constraints,  penalty  the  average  penalty 
score,  and  time  the  average  CPU  time  in  seconds.  The  better  results  between  the  two  algorithms  are  in 
bold. 

reasonable  performance,  in  the  rest  of  this  subsection  we  will  use  Dyna-WSAT(oip)  with  9  =  1/6  and 
(j)  =  1/5  as  default  parameters  in  our  experimental  analysis. 

As  mentioned  at  the  end  of  the  previous  section,  another  parameter  in  our  extended  WSAT(oip)  algo¬ 
rithm  is  the  number  of  moves  M  between  two  consecutive  check  points  for  examining  progress,  if  any, 
made  by  the  algorithm  in  extending  the  current  partial  assignment  to  a  complete  one  that  satisfies  the  max¬ 
imum  number  of  hard  constraints  under  the  current  aspiration  level.  A  good  value  for  parameter  M  can 
be  achieved  when  a  good  balance  is  made  between  the  algorithm’s  ability  to  find  satisfied  solutions  and  its 
ability  to  escape  from  local  minima.  This  leads  to  our  adaptive  parameter  approach,  in  which  the  parameter 
M  is  dynamically  adjusted  based  on  progress  made  or  not  made,  as  reflected  in  the  time  elapsed  since  the 
last  improvement  made  to  hard  constraints.  At  the  beginning  of  the  search,  we  give  an  initial  value  that  is 
proportional  to  the  number  of  hard  constraints  m  to  parameter  M.  If  the  number  of  hard  constraints  does 
not  decrease  over  the  last  M  search  steps,  we  increase  M  io  M  +  kim,  where  ki  is  a  positive  constant 
less  than  one.  Otherwise,  we  decrease  M  by  k2m,  where  /c2  is  another  positive  constant  less  than  one.  In 
our  experiments,  we  took  M  =  min{m,  AT/IO}  and  ki  =  k2  =  M/5,  where  K  is  the  maximum  number 
of  moves  for  a  restart. 

6.5  Applications  and  Experimental  Evaluation 

We  implemented  an  improved  and  extended  WSAT(oip)  algorithm  that  incorporates  the  backbone-guided 
biased  moves,  sampling-based  aspiration  search  and  dynamic  parameter  strategies.  We  shorthand  the 
improved  WSAT(oip)  as  EWSAT(oip).  In  this  section,  we  report  the  experimental  results,  comparing  the 
EWSAT(oip)  algorithm  with  its  predecessor,  the  WSAT(oip)  algorithm.  We  carried  out  our  analyses  on 
three  different  scheduling  problems:  our  crew  training  scheduling  problem  (Section  6.1),  progressive  party 
scheduling  and  basketball  tournament  scheduling.  The  last  two  problems  were  studied  in  [98],  and  also 
included  as  benchmark  problems  in  CSPEIB  [36],  an  online  repository  of  CSP  problems.  All  our  experi¬ 
ments  were  run  on  an  AMD  Athelon  1900  machine  with  2GB  memory. 

In  our  experiments,  we  tried  various  parameter  settings  for  the  WSAT(oip)  algorithm,  which  include 
the  probability  ph  of  choosing  a  hard  violated  constraint  over  a  soft  violated  constraint  and  the  size  of 
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tabu  list.  The  comparison  results  below  are  for  the  best  parameters  for  WSAT(oip).  Specifically,  on  our 
crew  scheduling  problems,  the  best  probability  ph  is  99%,  and  on  the  party  scheduling  and  basketball 
scheduling  problems,  the  best  ph  is  90%.  To  make  a  fair  comparison,  we  applied  dynamic  parameter 
method  to  automatically  adjust  the  noise  ratios  for  both  WSAT(oip)  and  EWSAT(oip),  and  let  EWSAT(oip) 
have  the  same  size  of  tabu  list  as  used  by  WSAT(oip),  which  was  set  to  4. 

6.5.1  Crew  training  scheduling 

The  first  and  main  problem  we  considered  is  our  crew  training  scheduling  problem.  The  test  set  consists  of 
six  large  and  many  small  problem  instances,  derived  from  a  real  application  domain  involved  with  a  large 
number  of  crew  members  of  different  specialty  and  various  equipment  that  requires  routine  maintenance. 
These  problem  instances  vary  in  sizes  and  degree  of  constrainedness;  the  largest  instances  has  79,580 
variables  and  143,896  constraints.  These  problems  were  collected  from  overconstrained  situations  and 
their  hard  constraints  did  not  seem  to  be  satisfiable  all  together.  Here  we  present  the  results  on  these  six 
large  instances. 

In  our  experiments,  we  allowed  WSAT(oip)  and  EWSAT(oip)  to  have  200  random  restarts  for  each  of 
their  run,  and  60,000  maximum  moves  (variable-value  changes)  with  each  restart.  The  average  results 
comparing  these  two  algorithms  over  40  runs  on  all  six  problem  instances  are  shown  in  Table  10.  We 
examined  the  average  minimum  number  of  unsatisfied  hard  constraints  (unsat),  the  average  minimum 
penalties  (penalty)  and  the  average  CPU  time  (time)  required  to  reach  solutions  of  such  qualities.  As  the 
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Problem 

WSAT(oip) 

EWSAT(oip) 

name 

n 

m 

median 

average 

median 

average 

ppp:l-12,16 

4662 

31725 

0.175 

0.185 

0.186 

0.190 

ppp:l-13 

4632 

30964 

0.406 

0.441 

0.388 

0.423 

ppp:l,3-13,19 

4608 

30348 

0.469 

0.472 

0.388 

0.449 

Ppp:3-13,25,25 

4644 

31254 

0.625 

0.713 

0.656 

0.718 

ppp:l-ll,19,21 

4602 

30179 

15.283 

15.814 

9.453 

12.856 

ppp:l-9, 16-19 

4626 

30747 

44.906 

63.553 

34.546 

58.302 

Table  11:  Comparison  on  progressive  party  seheduling  problem,  where  n  and  m  are  the  numbers  of 
variables  and  elauses,  respeetively,  and  median  and  average  are  the  median  and  average  CPU  times  in 
seeonds.  The  better  results  are  in  bold. 


results  show,  EWSAT(oip)  is  able  to  find  better  solutions  with  more  hard  eonstraints  satisfied  and  lower 
penalfies  for  all  problem  insfanees,  and  some  limes  wilh  less  exeeulion  time. 

Additional  insights  were  gained  when  we  examined  the  anytime  performanee  of  the  two  algorithms  on 
these  diffieult  problems.  Figure  5 1  shows  sueh  an  anytime  eomparison  on  one  of  the  six  instanees,  also 
averaged  over  40  runs.  As  shown,  EWSAT(oip)  ean  make  signifieant  improvement  in  an  early  stage  of  the 
seareh,  indieating  that  it  explores  more  fruitful  regions  of  the  seareh  spaee  more  effeetively  than  WSAT(oip). 
The  anytime  results  on  the  other  problem  instanees  were  similar  to  that  in  Figure  5 1 . 

6.5.2  Progressive  party  scheduling 

The  Progressive  Party  Problem  (PPP)  is  to  progressively  timetable  a  sequenee  of  parties.  There  are  a 
total  of  six  eritieally  eonstrained  PPP  problem  instanees  in  CSPEIB  [36],  whieh  are  all  satisfiable.  We 
eonsidered  all  of  them  in  our  experiments.  We  allowed  100  random  restarts  in  one  run  of  the  algorithm, 
eaeh  of  whieh  used  60,000  moves.  We  averaged  the  results  over  40  runs  for  eaeh  of  two  algorithms. 
The  problem  sizes  are  relatively  small,  eomparing  to  the  erew  seheduling  problems,  with  the  number  of 
variables  less  than  five  thousand  and  the  number  of  eonstraints  no  more  than  32,000.  Table  1 1  shows  the 
median  and  average  CPU  times  to  reaeh  satisfying  solutions  by  WSAT(oip)  and  EWSAT(oip). 

On  four  of  the  six  problem  instanees,  both  algorithms  take  less  than  one  seeond  to  hnish  and  have 
similar  performanee.  On  the  other  two  problem  instanees,  EWSAT(oip)  ean  reduee  median  exeeution  time 
from  15.3  seeonds  to  9.5  seeonds  and  44.9  seeonds  to  34.6  seeonds,  giving  time  reduetions  of  37.9%  and 
22.9%,  respeetively. 

6.5.3  Basketball  tournament  scheduling 

The  Atlantie  Coast  Conferenee  (ACC)  Basketball  Seheduling  Problem  is  to  arrange  a  basketball  tournament 
in  the  ACC.  The  problem  was  originally  deseribed  by  Miehael  Triek  and  George  Nemhauser.  Walser 
developed  a  pseudo  Boolean  integer  linear  programming  model  for  these  problems  [98].  The  objeetive  of 
an  eneoded  ACC  seheduling  problem  is  to  satisfy  all  the  hard  eonstraints  while  minimizing  the  total  penalty 
eaused  by  violated  soft  eonstraints.  The  difheulties  of  the  available  problem  instanees  vary  dramatieally. 


100 


Problem 

WSAT(oip) 

EWSAT(oip) 

name 

n 

m 

median 

average 

median 

average 

acc-tight:2 

1620 

2520 

0.86 

1.03 

0.77 

1.05 

ace-tight:  3 

1620 

3249 

1.30 

2.16 

1.26 

1.83 

acc-tight:4 

1620 

3285 

44.14 

61.49 

35.35 

48.69 

acc-tight:5 

1339 

3052 

1171.17 

1071.18 

603.57 

609.05 

Table  12:  Comparison  on  ACC  backetball  scheduling  problem,  where  the  legends  are  the  same  as  in 
Table  11. 


Here  we  only  consider  four  instances  of  moderate  difficulties. 

The  experiment  setup  was  the  same  as  for  the  party  scheduling  problem  considered  earlier.  The  median 
and  average  times  to  reach  satisfying  solutions  to  these  problems  are  included  in  Table  12.  EWSAT(oip) 
outperformed  WSAT(oip)  on  all  these  instances  except  the  slightly  slower  average  time  on  instance  acc- 
tight:2.  The  performance  of  EWSAT(oip)  seems  to  improve  on  hard  instances.  Eor  example,  EWSAT(oip) 
reduced  the  median  running  time  by  48.4%  on  acc-tight:5,  increased  from  20.5%  on  acc-tight:4. 

6.6  Conclusions 

WSAT(oip)  is  an  extensively  applied  integer  local  search  algorithm  for  solving  constraint  problems  with 
hard  and  soft  constraints  which  are  represented  as  overconstrained  integer  linear  programs  (OIPs).  In 
this  section,  we  introduced  three  strategies  to  improve  the  performance  and  applicability  of  WSAT(oip)  in 
solving  complex  scheduling  problems:  biased-move  strategy  to  improve  the  efficacy  of  local  search  by  ex- 
ploifing  backbone  sfrucfures;  sampling -based  aspirafion  search  fo  hnd  high  qualify  solufions  and  improve 
anyfime  performance;  dynamic  paramefer  adapfafion  fo  make  WSAT(oip)  robusf  and  more  applicable  fo 
real-world  problems.  Our  experimenfal  resulfs  on  fhree  large  and  complex  scheduling  problems  show  fhaf 
our  improved  WSAT(oip)  algorifhm  signihcanfly  improves  upon  fhe  original  WSAT(oip)  by  Ending  beffer 
solufions  on  overconsfrained  problems  or  finding  beffer  or  same-qualify  solufions  sooner.  We  expecf  fhaf 
fhese  new  mefhods  can  be  applied  fo  ofher  search  algorifhms  and  combinaforial  opfimizafion  problems. 

The  mosf  imporfanf  lesson  we  have  learnf  from  fhis  research  was  fhaf  single  mefhod  may  nol  be  effec- 
five.  If  usually  requires  fo  combine  many  differenl  sfrafegies  in  order  fo  solve  difhcull  problem  insfances 
wifh  a  reasonable  amounf  of  time. 
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